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MANIFESTO 


Since 2005, Digicult is one of the main online platform that examines the impact of digital technologies and science on 
art, design, culture and contemporary society. Based on an international Network of critics, curators, artists, designers, 
professors, researchers, journalists, media center, galleries and festival, Digicult is an editorial project that daily publish 
news, informations, articles, interviews, reports and even essays, artists’ book and the Digimag Journal through its online 
publishing service Digicult Editions. 

Digimag Journal is an interdisciplinary online publication seeking high-standard articles and reviews that focus on the 
impact of the last technological and scientific developments on art, design, communication and creativity. Following the 
former Digimag Magazine (72 issues in 7 years), it is based on international call for papers on given subjects and provides 
readers with comprehensive accounts of the latest advancements in the international digital art and culture scene. 

Digimag was born as a monthly magazine and published 72 issues in over 7 years. It provided readers with comprehensive 
accounts of the latest advancements in the international digital art scene and culture. The magazine evolved year after 
year, issue after issue, morphing into a hybrid instrument able to reflect the complexity of contemporary artistic and cul¬ 
tural production. It quickly became a cultural instrument, a tool for academics, researchers, students, artists, designers, 
geeks and practitioners connected to the main international media centers, universities, contemporary art galleries, digital 
art festivals and hacktivist networks. 

Digicult Editions is the publishing initiative of the Digicult project, whose goal is to be active in the publication of the 
Digimag Journal, but also critical and theoretical books and essays commissioned to international authors, university 
thesis of special interest, publications edited in collaboration with other national and international publishers, conference 
proceedings and classes materials connected to educational activities, as well as peer-reviewed publications with institu¬ 
tional partners. 

Digicult Editions has now 4 different book series: Networks (Internet, Networks, IT), (h)activism (social and political im¬ 
pact of technologies). Arts and Sounds (more strictly visual and/or sound art). Rhizomes (genres’ intersection) and Digital 
narratives (new narrative formats: literature, gaming etc). 

Digicult Edition uses all the tools of a contemporary digital publishing: the print on demand (POD) approach through 
Peecho, Epub and Mobi formats, always giving the chance to join all the prewieves through the Digicult Library on 
Issuu https://issuu.com/home/publisher). All contents by Digicult Editions are circulating under CC Licences: Attribu- 
tion-NonCommercial-ShareAlike 3.0 
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SMART 

MACHINES FOR 
ENHANCED ARTS 


Artificial Intelligence (AI) and Machine Learning (ML) might be considered by many as synonyms, also because they are 
the buzzwords of this decade. But actually they are not. They both question though, the ability of the machines to perform 
and complete tasks in a “smart” way, challenging human intelligence and specificity. 

With machines becoming more and more intelligent. Machine Learning is nowadays not only an interesting and challen¬ 
ging topic, but also a crucial discipline. If initially computing was just a matter of calculations, now it has moved beyond 
simple “processing” and implies also “learning”. In the age of Big Data and loT, machines are asked to go beyond pure 
programming and algorithms procedures, introducing also predictions of data, OCR and semantic analysis, learning from 
past experiences and adapting to external inputs, reaching out the domain of human productions and processes. 

As Gene Kogan and Francis Tseng write in their in-development book “Machine Learning for Artists”, we can “pose 
today to machines a single abstract problem: determine the relationship between our observations or data, and our desired 
task. This can take the form of a function or model which takes in our observations, and calculates a decision from them. 
The model is determined from experience, by giving it a set of known pairs of observations and decisions. Once we have 
the model, we can make predicted outputs””. 

So, the subject of Machine Learning and Artificial Intelligence methods more in general, are going thusly much further 
the technology or science fields, impacting also arts, product design, experimental fashion and creativity in general. As 
ML features can fit with digital arts practices, we’re lead to explore the way some AI techniques can be used to enhance 
human performative gestures and creativity models. 

How biological systems and machine intelligence can collaborate to create art, and which is the cultural outcome for 
our society? Which is the new role of creativity in this scenario? How the contemporary will face a future generation of 
automated artificial artists/designers, able to learn from the creatives themselves, or to have a direct impact on human 
creativity? Will the anthropocentric vision of the creative process behind the artistic creation, affected by new intelligent 
Neural Networks? 
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Louis Bee. Istione Sarksi, 1997-99, artificial life model, softimage 


We observe the success of artificial neural networks in sim¬ 
ulating human performance on a number of tasks: such as 
image recognition, natural language processing, etc. How¬ 
ever, There are limits to state-of-the- art A1 that separate 
it from human-like intelligence. Humans can learn a new 
skill without forgetting what they have already learned 
and they can improve their activity and gradually become 
better learners. Today’s A1 algorithms are limited in how 
much previous knowledge they are able to keep through 
each new training phase and how much they can reuse. In 
practice this means that it is necessary to build and adjust 
new algorithms to every new particular task. This is closer 
to a sophisticated data processing than to real intelligence. 

This is why research concerning generalisation are becom¬ 
ing increasingly important. Processes such as intuition, 
emotions, planning, thinking and abstraction are a part of 
processes, which occur in the human brain. Abstraction al¬ 
lows for making analogies, coding relations and relations 
between relations. Generalization is a process in which the 
brain observes that a certain fact referring to a set of ob¬ 
jects, refers to a greater set of objects. Processes occurring 
in the brain have an extremely plastic and dynamic char¬ 
acter and cannot be reduced to one basic construction and 
operation. Many processes have very distributed character, 
for instance memories are not located in a particular place; 
the brain has holographic character [Pribram 1991]. A spe¬ 
cial role of a kind of creators is played by random processes, 
which allow for collision and splitting of structures, and 
leaps between different levels of generality. A generaliza¬ 
tion in A1 means that system can generate new composi¬ 
tions or find solutions for new tasks that are not present in 
the training corpus. 

General Neural Model and intelligent agent should have 
very general learning capabilities, should not just be able to 
memorize the solution to a fixed set of tasks during creat¬ 
ing of stories, but learn how to generalize to new problems 
it encounters. It can generalize problem in the sens that 
solving one or more of tasks should make solving other 
task easier. There is domain called AGl where will be pos¬ 
sible to find solutions for this problems. Artificial general 
intelligence (AGl) describes research that aims to create 
machines capable of general intelligent action. “General” 
means that one AI program realizes number of different 
tasks and the same code can be use in many applications. 
We must focus on self-improvement techniques e.g. Re¬ 
inforcement Learning and integrate it with deep learning, 
recurrent networks, etc. 


RECURRENT NEURAL NETWORKS 

Models of sequential data, such as natural language, speech 
and video, are the core of many machine learning applica¬ 
tions. Recurrent Neural Network is a powerful model that 
learns temporal patterns in sequential data. A recurrent 
neural network (RNN) is a class of artificial neural net¬ 
work where connections between units form a directed 
cycle, meaning that Recurrent Neural Network contains 
feedback connections, connections from any unit to it¬ 
self. This allows it to exhibit dynamic temporal behavior. 
Unlike feedforward neural networks, RNNs can use their 
internal memory to process arbitrary sequences of inputs. 
This makes them applicable to tasks such as handriting re¬ 
cognition, speech recognition and after modification as a 
performative and/or compositional tool for composer and 
musicians. 

Greating of feedback in RNN provides interesting creat¬ 
ive posibilities, recurrent neural networks can evolve to a 
unstable states and they can create chaotic or random out¬ 
puts. Ghaotic behavior of recurrent neural networks has 
been observed before eg. by Maas (Maas et all, 1990). It 
was shown that smooth variation of one of the paramet¬ 
ers of the original map gives rise to period-doubling bi¬ 
furcations. Feedback and chaotic behavior of RNN causes 
that some artists and composers are starting to use RNN 
in their artistic work. For example GTRNN Gontinuous 
Time Neural Networks are implemented in modular ex¬ 
tensible computer music platforms such as Supercollider, 
Pure Data, MaxMsp. Sound or video parameters can evolve 
and be formed by using Recurrent Neural Networks. 

RNN models can be uses to tasks such as handwriting 
recognition, speech recognition, natural language pro¬ 
cessing, video recognition, etc. Natural language model¬ 
ing has been widely studied in the past (Goodman, 2001b; 
Young et al., 1997; Koehn et ah, 2007). In particular, mod¬ 
els based on RNN have been very successful in automatic 
speech recognition (Dahl et ah, 2012), language modeling 
(Mikolov, 2012) and video classification (Simonyan & Zis- 
serman, 2014). 

RNN represents time recursively. For example, in the 
simple recurrent network, the state of the hidden layer at a 
given time is conditioned on its previous state. This recur¬ 
sion allows the model to store complex signals for arbitrar¬ 
ily long time periods, as the state of the hidden layer can be 
seen as the memory of the model. However there is a prob- 
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CTRNN in action 


lem of learning long term patterns in Recurrent Neural 
Networks. Recurrent networks was difficult to train using 
simple optimizers, such as stochastic gradient descent, due 
to the vanishing gradient problem. For example the sig¬ 
moid function have a gradient which is close to zero al¬ 
most everywhere or the gradient can be backpropagated 
through time, its magnitude is multiplied over and over 
by the recurrent matrix. If the eigenvalues 
of this matrix are small (i.e., less than one), 
the gradient will converge to zero rapidly. 

Learning longer term patterns in real data, 
such as in natural language, is possible us¬ 
ing gradient descent by using a structural 
modification of the simple recurrent neural 
network architecture. Many modifications 
were proposed to deal with the vanishing 
gradients eg. the long short term memory 
(LSTM) recurrent neural network (Ho- 
chreiter & Schmidhuber, 1997) is a modi¬ 
fied version of recurrent network with gates which con¬ 
trol flow of information to hidden neurons.This allows the 
network to potentially remember information for longer 
periods. 

Most of the research on the use of AI in interactive ap¬ 
plications concerns computer games, beginning with tra¬ 


ditional two-player adversarial games like tic-tac-toe and 
extending to modern strategy games. This type of research 
however, has a limited application in storytelling or art, be¬ 
cause the goal of AI agents in these games is to maximise 
reward, which often fails to advance the narrative threads 
and almost entirely overlooks the creation of interesting 
scenarios or compositions. Magerko (2009) conducted re¬ 
search with theatre performers to explore 
how to create scenes in real time without 
a preexisting scenario. Unfortunately, the 
basic conclusion of this research is that the 
actor should act on the basis of a huge set 
of initial scripts, which contain expecta¬ 
tions as to what people do in different scen¬ 
arios. There have been several attempts to 
implement the above approaches through 
introducing text into the game, building 
chatbots or intelligent assistants. However, 
these approaches have a limited scope, be¬ 
cause they do not encompass the numerous phenomena 
known in natural language, such as the creation of meta¬ 
phors, analogies and generalisations, which are crucial for 
human thinking and for creating stories. If the AI program 
works in order to create a story, it must be prepared to un¬ 
derstand everything that the human might think and must 
be able to communicate in a natural language. Neural lan- 


HOWEVER THESE 
APPROACHES HAVE 
A LIMITED SCOPE, 
BECAUSE THEY DO 
NOT ENCOMPASS 
THE NUMEROUS 
PHENOMENA 
KNWON IN NATURAL 
LANGUAGE 
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guage models have garnered research interest for their ability to learn complex 
syntactic and semantic representations of natural language (Sutskever et al., 
2014). Recurrent Neural Network (RNN) is a powerful model that learns tem¬ 
poral patterns in sequential data. Wen et al. (2015) proposed a RNN approach 
to the generation of utterances from dialog acts but their system requires one to 
pre-process the data. 

DEEP REINEORCEMENT LEARNING 

Thinking about programs in terms of binary codes and about functioning of the 
brain in terms of self-improvement through optimization of the codes, we may 
perceive the search for brain model as the search for best learning algorithms, 
and as an attempt of creating best predictions. The most interesting AI method 
is Reinforcement Learning. The brain works without deus ex machine, the rule 



of its organization is the rule of the 
shortest description, which allows for 
choosing shortest models of reality. 
A program is a set states that repres¬ 
ents a given situation and the set of 
operators (actions) and which allows 
for transition between states. A gen¬ 
eral framing of the space of transition 
between states is the Hidden Markov 
model, in which every transition has a 
certain probability of occurrence. Real 
states of the world are unknown, they 
can only be approximated. We obtain 
data through sensors from the envir¬ 
onment. The data is recorded in the 
form of vectors. Markov models are 
static: the agent is unable to model his 
actions, he cannot change the world 
he is in. An extension of this model 
are Markov Decision Processes. MDP 
generalizes Markov models by intro¬ 
ducing additional possibilities: con¬ 
sequences of actions may not be know 
a priori, even if the consequences of 
actions are known, their value is un¬ 
known, the value of the action is dif¬ 
ficult to predict, because the reward 
is often delayed. In this situation the 
model of best actions is not known, 
but it has to be discovered. The agent 
uses certain actions and analyses their 
results. The actions which bring re¬ 
ward are not known, but they have to 
be discovered by trial and error. The¬ 
ories that apply the above method are 
called models of reinforcement learn¬ 
ing. Models of reinforcement learning 
allow the agent to choose adequate de¬ 
cisions on the basis of exploration of 
the environment that changes dynam¬ 
ically. By examining the space of states 
that bring reward, the agent may learn 
from the history of his previous ac¬ 
tions. The most interesting situation 
occurs when the space of states is only 
partially observable. Reinforcement 
learning works, because the agent can 
make local improvements in order to 
increase the reward. 
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Quantum Enigma by RBL (using radioactive decay) 


A deep neural network (DNN) is an artificial neural network with multiple hid¬ 
den layers between the input and output layers. Each successive layer uses the 
output from the previous layer as input. It is system of multiple layers of nonlin¬ 
ear processing units that learns of feature representations in each layer and form 
a hierarchy from low-level to high-level features. Deep learning networks can 
be applied to any problem for example in language, sound or image processing. 
Deep and recurrent neural networks are powerful models that achieve high 
performance on difficult pattern recognition problems in vision, and speech 
(Krizhevsky et ah, 2012; Hinton et ah, 2012; Dahl et ah, 2012). Reinforcement 
Learning can be used to improve dialogue managers, e.g. for transitions between 
dialogue states (Rieser and Lemon, 2011), for non-goal-orientated dialogues (Li 
et ah, 2016), for hot-hot dialogues and for inventing new languages by agents 
(Das et ah, 2017). 

Deep Learning is becoming more and more popular method. Commercial using 
of Deep Learning models is often associated with using data from massive data 
centers eg. Google, Lacebook and it is difficult to verify if used model is really 
intelligent and can generalize knowledge or if it is only sophisticated complex 
automated system that uses the brute force method based on unlimited access 
to data generated by users. The application of deep learning in art looks unin¬ 
teresting because constructed networks simulate only human behaviors; in this 
case, they use art history databases to generate objects that imitate artworks 
from the past. 


into neural networks. Both of these 
approaches are perfectly complement¬ 
ary because there is no interesting 
self-improvement system without the 
clever use of random generators and 
vice versa. 

KINGDOM OL RANDOMNESS 

To obtain interesting results in music 
and art we need randomness. Ran¬ 
domness is important when you want 
the Neural Network from the same 
input to create different possibilit¬ 
ies as an output, without generating 
the same output over and over again. 
Therefore it is different than in sci¬ 
ence, which is all about grouping and 
clustering. In art and music we don’t 
want to endlessly obtain the same res¬ 
ult. When you are composing sound 
or images, you don’t want the neural 
network program to create the same 
sets of sounds and images; instead, 
you need creativity and variability. 
One of the solutions is to parametrize 
NN outputs with the use of probabil¬ 
ity distributions. A different way is to 
add noise directly to NN, instead of 
modeling the distribution paramet¬ 
ers. Paradoxically, in such NNs, the 
more randomness during training, 
the better the results. Good random 
generators allow to avoid situations, 
when Neural Networks gets fixed in 
local minima. The importance of well 
designed random initialization and 
momentum in deep learning was ob¬ 
served for example by Sutskever and 
Hinton (2016). 


Therefore, if we want to make the next shift (challenge) we have to put more 
emphasis on research concerning self-improvements of system eg. different 
types of algorithms associated with reinforcement learning and perform inter¬ 
esting fusions of reinforcement learning with deep learning. The goal is to in¬ 
corporate the reinforcement learning process into deep learning for creating a 
system that will have an ability to learn and self-improve. Another way to do 
this is to study the methods connected with randomness and to integrate them 


Random number generators are ap¬ 
plied in many domains for instance 
in music programs, computer games 
etc. In numerous compilers random 
number generators are used as ordin¬ 
ary functions. The problem with these 
generators however is that they never 
produce random numbers, not even 
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numbers that approximately appear to be random. Limits 
of our perception makes us consider sequences of numbers 
they produce as random, and maybe in the case of games of 
chance this ’randomness’ is sufficient, but in cryptography 
it is hopelessly not random and completely useless. No ran¬ 
dom number generator build with the use of computer (or 
abstract Turing machine) will generate random numbers. It 
is impossible to obtain a sequence of random numbers with 
the use of computer. Computers are entirely deterministic 
machines: we put something in, subsequently we subject 
it to entirely predictable operations and receive something 
’’new”. If we put the same thing into the computer again (in 
different times) we will get exactly the same result. If we 
put the same input sequence into two different computers, 
we will also gain the same result. The number of states of a 
computer is finite. 

The result is completely determined by input data and 
functions we use. Every random number generator build 
with the use of computer is, by definition periodic. Of 
course, such periodic generator is predictable. If it is pre¬ 
dictable it can’t be random. A real random number gener¬ 
ator requires a random input. A computer 
does not have random input. The only pos¬ 
sibility when it comes to randomness in 
case of computer science techniques is the 
creation of pseudorandom number gener¬ 
ators (PRNG). They can be useful in some 
applications if the period of the obtained se¬ 
quence of numbers is long enough, which 
means that numbers will repeat but after a 
relatively long time. The beginning of this 
sequence should consists of numbers that remind of ran¬ 
dom numbers. Many pseudorandom generators have been 
built, which are periodic, but the potential periods have the 
length of 2256 bites. However even generators with very 
long periods create strange correlations between numbers. 
Unfortunately every deterministic generator will produce 
them if it will be used in a specific way, but there are ways 
of minimizing the number of correlations. Structure of 
PRNG consists in using a certain number and recurrence 
function. We start with a certain number (seed). 

Then we subject it to mathematical function. We obtain a 
number, which is again subjected to the same function. We 
repeat the procedure. We obtain pseudorandom sequence 
of numbers. The process is constructed in such a way so 
that the numbers repeat after some time. How fast it will 
occur depends on the function used. At some point PRNG 
will produce the number from which we started. From this 


moment the numbers will repeat periodically. In weak gen¬ 
erators the periods are short, in good ones very long, which 
means that the sequence of numbers may repeat after mil¬ 
liards of operations. A choice of good, incompressible seed 
has a crucial importance in cryptography and art. When 
the secret cypher key is generated in a pseudo-random 
manner, if we get the seed, we also get the key. In cryp¬ 
tography we need really random number sequences. This 
means sequences, which can’t be repeated: it is impossible 
to repeat a sequence of bits produced by such generator. 
This problem can be examined on the example of creating 
a key. Keys should be generated in such a way, so that the 
sequence used is really random. Otherwise, the adversary 
may get the copy of the key generator and break the given 
cryptographic system. 

Random number generator should be subjected to tests, 
which consist in attempts of compressing generated se¬ 
quence. One of the popular methods of generating random 
numbers is obtaining bits from computer clock. It doesn’t 
however guarantee good quality randomness because com¬ 
puters use many mechanisms of clock synchronization. 

One shouldn’t use too many bits generated 
through this method because using the 
same procedure several times effects in the 
emergence of correlations, which are easy 
to find. Randomness obtained from meas¬ 
uring time when the keyboard remains idle 
is another popular method, which consists 
in measuring short time intervals between 
keyboard presses. However a fundamental 
flaw of this method is that the generated 
keys are usually very short. A good way of obtaining big 
number of random numbers is to use bits from the sur¬ 
rounding world, for example using atmospheric noise. 
This method requires specialized measuring devices that 
allow for measuring time between events. 

Generators that use thermal noise (e.g. from semicon¬ 
ductor material), as well as those that use computer disk 
drive, and measure time necessary for reading set of data 
from the disk have been created. Air turbulences influence 
variations in the rate of disk rotation. There are also other 
methods of obtaining numbers from noise, which consist 
in measuring the position of the machine, mouse, screen 
behavior, components of currently displayed image, GPU 
loads, microphone signal etc. Fundamental flaw of these 
methods is that there may occur some repeatable correl¬ 
ations introduced by measuring devices. These devices are 
often synchronized in order to enable correct, repeatable 


WHEN THE SECRET 
CYPHER KEY IS 
GENERATED IN A 
PSEUDO-RANDOM 
MANNER, IF WE GET 
THE SEED, WE ALSO 
GET THE KEY 
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activity of the computer. A very good method is using ra¬ 
dioactive material and Geiger counter in which, during ra¬ 
dioactive decay the time between following clicks is always 
different. Randomness from the quantum level obtained 
from radiation emitted during decay of radioactive mater¬ 
ial has an extremely good quality. One may obtain a big 
number of random bits from Geiger counter and use it as 
a key. We use a sequence of the length of for instance 256 
bits as an input of one-way hash function. Hash function 
is a mathematical function that transforms a sequence into 
a sequence of a precise length. We use one-way function, 
which means that it is easy to count the shortened sequence 
on the basis of input sequence but it is impossible to do it 
the other way round. Quantum mechanics claims that in the 
real world, randomness occurs in a pure form. In quantum 
states there is fundamental randomness and it cannot be 
changed. If we interpret the formulas of quantum mech¬ 
anics we can say, that elementary particles do not exist in 
any precise location. They exist in many locations simul¬ 
taneously, with a certain probability of occurrence. 

QUANTUM ENIGMA consists of installation, action and 
program, whose goal is to extend the notion of art by deal¬ 
ing with randomness. I reveal the process of radioactive 
decay as the best and strongest manifestation of the idea 
of randomness. Thorium is a medium of my idea: I have 
chosen Thorium as radioactive material, which is the 
source of fundamental random processes and is not asso¬ 
ciated with art. Thorium is a source of dispersion and real 
disorder. The real performative element contained in the 
action reveals pure process of decay and energy emission. 

MIND UPLOADING 

Mind Uploading (MU) is the process of copying the brain 
from natural substrate to artificial one. MU is carried out 
through scanning the structure of a brain and building an 
appropriate informatics model, which is true to the original 
and after launching it on an adequate hardware, behaves 
like the original brain. By emulation we understand a 1 to 
1 model, in which all properties of the system have been 
retained. Emulation, copying and modifying of the brain 
will bring explosion of diversity. We are very good at cre¬ 
ating maps of the human brain in different scales (we have 
increasingly better techniques of scanning brain structures 
in different scales), however our informatics brain mod¬ 
els are supremely primitive. There are interesting con¬ 
sequence of building MU in the future. People will become 
more heterogenous, more varied in terms of physical and 
cognitive dimensions. Another consequence will be the 


change of value system: life will become less valuable. Life 
will become ’’cheap” because we will be able to emulate it. 
Why not participate in a risky game, when in a worst case, 
I can activate the backup copy of myself. The belief in ’’im¬ 
mortality” may change our behavior. Analogy: in computer 
game we value our life less than in real life, because we can 
reset it anytime. After the mind up- loading turning point, 
freezing, copying, slowing down and modifying of the 
brain will become acceptable. Qf course there will be other 
consequences: ethical, political, economic, medical, reli¬ 
gious and cultural. I highlighted value of life, because it has 
influence on other values. Qther consequences of MU are 
connected with the notion of identity and consciousness. 
Gan human consciousness be emulated and transferred? 
Does the transfer preserve consciousness? People are con¬ 
scious, which means that they have conscious, subjective 
experiences. They lie at the center of our mental life and 
give our life meaning. If we loose consciousness, in a signi¬ 
ficant sense we cease to exist. If brain emulation is devoid 
of consciousness, then it probably doesn’t exist at all, at the 
very least it is zombie existence. The problem is complex 
because our understanding of consciousness is very weak, 
we do not know how brain’s activity makes consciousness 
possible. 

■ ROBERT LISEK 
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AN UBIQUITOUS INFLUENCE. 
INTRODUCTION TO Al AND ML 


by Filippo Lorenzin 


What is an Artificial Intelligence? 
How can a machine learn new 
strategies to accomplish a task? Al¬ 
though these questions are not new or 
original, in the last months they have 
been addressed by a number of experts 
whose backgrounds spanned from art 
anthropology to engineering, from 
biology to poetry. 

Art festivals, exhibitions and pan¬ 
els all around the world often tried 
to question what is the future of this 
field while sometimes forgetting to 
look at what’s the current situation; it 
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can be hard to not dream about what 
may be achieved in the next few years 
when some of the recent most com¬ 
mented political events had been af¬ 
fected by algorithms and we’re aided 
24/7 by corporate digital assistants 
like Amazon’s Alexa and Apple’s Siri 
- without forgetting to mention more 
subtle but not less effective systems 
like the algorithms behind the cus¬ 
tomized suggestions we receive every 
day on social media, search engines 
and online shopping platforms. The 
everyday sophisticated influence exer¬ 
ted by Artificial Intelligence and Ma- 
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chine Learning is perhaps the reason 
behind the renovated interest in what 
has been considered since not long 
time ago a sci-fi dilemma rather than 
an urgent question with potential 
deadly and utopian implications. 

Artists questioned what is the value of 
building a creative machine paying at¬ 
tention to what degree the very same 
meaning of “creativity” can depend on 
cultural backgrounds. Digicult, alert 
to the hottest debates regarding the 
blurred line between art and science 
since its foundation in 2005, took 
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the opportunity of the publication of 
this journal to ask a selected group 
of experts to illustrate the directions 
of present studies in Artificial Intel¬ 
ligence and Machine Learning. We 
contacted a wide range of figures in 
order to reflect the ubiquitous impact 
of such technology in art, science and 
marketing - their considerations don’t 
take a positivistic stance for granted 
but rather suggest the intricacy that 
surveys about A1 and ML are rising 
right now. 


Caroline Sinders | Machine learning 
designer/user researcher, artist and 
digital anthropologist 

The state of contemporary Machine 
Learning Research is having a resur¬ 
gence, but with a focus on actionable 
product design within software. The 
last resurgence of interest in A1 and 
machine learning was in the 1980s but 
the scope of technology and products 
was so different then; it was so hard¬ 
ware focused and so focused on what 
the code could do. What I see now is 


a much more specific focus on how 
products can and will utilize machine 
learning. Products, like Netflix and 
Pinterest, from recommendation soft¬ 
ware to apps, to more general soft¬ 
ware capabilities with search or ana¬ 
lytics, and APIs as a service, are using 
various forms of machine learning as 
an integral part of the product design 
process and product experience. This 
feels like the age of machine learning 
creating more “future tech” in design, 
specifically. 

What we will see is the rise of the 
more technical designer, and the sys¬ 
tems designer, over the aesthetic de¬ 
signer, with a background in ethics, 
usability, and data visualization. A 
major issue facing machine learning, 
as a field, is that it is such a technical 
space, and to properly utilize machine 
learning and make good, accurate, 
innovative products that work, you 
need technologists, producers, de¬ 
signers, and c-levels who understand, 
at some level, what machine learning 
can do. That doesn’t mean a designer 
needs to code but what is needed is a 
designer with a background in under¬ 
standing technology, data, and ethics, 
can imagine, scope out, and innovate 
on a problem from a product and UX 
standpoint. This will be about creat¬ 
ing a fluency of UX, ethics, and data 
visualization and baking that into the 
design pipeline from the algorithmic 
creation to the product creation. 

Mario Klingemann | Artist in resid¬ 
ence at the Google Cultural Institute 
working with algorithms and data 

As an artist who tries to incorporate 
the latest research and findings in 
the field of machine learning into my 
practice, I couldn’t be more pleased 
with the culture of open research, 
knowledge and particular code shar- 
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ing that has established itself increas- 
ingly over the past years. Without 
the combination of publicly available 
datasets, shared pre-trained models 
and well-maintained frameworks we 
would not have seen this rapid pro¬ 
gress of novel neural architectures and 
algorithmic advancements. Of course, 
with this boom comes also the hype 
with all its increased noise and over¬ 
inflated expectations. On the positive 
side the increased public and media 
interest in machine learning and par¬ 
ticular “Al” might help with getting 
funding much easier, on the negative 
side there seems to be more derivative 
or repetitive research being done that 
does not really add to the knowledge 
base but rather makes the truly innov¬ 
ative ideas harder to discover. In our 
attention economy, papers that know 
how to sell themselves visually or that 
are using popular data sets like cats 
or “famous art” bubble go to the top 
easier than a research that does not 
try to appeal to non-academics or the 
media. 

Now a basic classification has been 
solved and neural generation is in a 
stage where classification was 5 years 
ago. As a visual artist, of course, I 
hope the bar for high-resolution out¬ 
put will be raised continuously but 
seeing the progress in that area I have 
no worries there. The biggest missing 
piece in all of this is what I would call 
“storytelling” or the understanding 
and generation of the long term struc¬ 
ture. Once we are able to find a way 
to understand and encode “meaning” 
I believe that everything else can be 
derived from that - be it text, music 
or the generation of visuals. My guess 
is that the impulses to solve this will 
come from the digital humanities and 
the field of game development. In the 
short term, attention mechanisms 
like the Transformer architecture 


by Google research will gain a lot of 
traction. Lastly, I can imagine seeing 
reinforcement learning being applied 
to the internet as a whole: if you treat 
a browser like a game engine and the 
goal of the game is to answer a ques¬ 
tion or solve a problem by search¬ 
ing the combined knowledge of the 
world, it sounds almost like general 
intelligence to me. 


David Seamon | Environment-be¬ 
havior researcher 
and Professor of Ar¬ 
chitecture at Kansas 
State University in 
Manhattan, Kansas 


and finding better ways to bring those 
algorithms to current hardware, es¬ 
pecially graphics processing units 
(GPUs). Now instead of GPU chips 
rendering millions of pixels for games 
and visualization environments. Ma¬ 
chine Learning is taking advantage of 
this hardware to produce faster train¬ 
ing and evaluation of machine learn¬ 
ing models. Gurrent algorithms allow 
machines to learn rapidly without hu¬ 
man intervention. 


To me, technology 
remains a tool, but 
we are moving into a world where 
the tool is becoming an integral part 
of human worlds and thereby shapes 
those human worlds rather than serves 
them. Technology is no longer a tool 
but a taken-for-granted medium that 
makes lifeworlds qualitatively differ¬ 
ent than worlds before around 2000. 
The destruction of punctuation and 
spelling are one simple example, but 
the mobile phone and the virtual-real¬ 
ity devices begin to shape a world that 
seems to move human beings toward 
more and more passive states and 
actions. Understanding this shifting 
state of human being and asking what 
it means for important human qualit¬ 
ies like care, respect, obligation, com¬ 
munal ties, and so forth is to me the 
most important question. 

Robots and Pencils | Mobile strategy 
and app development firm known 
for its top-ranking consumer apps 
and transformative enterprise apps 

Gontemporary Machine Learning Re¬ 
search is primarily the combination 
of finding new training algorithms 


THE 

IMPLEMENTATION OF 
MACHINE LEARNING 
IN THE MODERN 
WORLD WILL BE 
UBIQUITOUS 


Today, Machine 
Learning not only in¬ 
volves understand¬ 
ing content, such as 
speech recognition or 
translation, but also 
creating content. Pro¬ 
jects like Magenta and Deepjazz, for 
example, generate music from scores 
of samples. Google’s DeepDream can 
generate digital art by enhancing pat¬ 
terns in images using a convolutional 
neural network. Machine Learning 
Research has become one of the most 
studied fields of research and will un¬ 
doubtedly impact all areas of society. 
Research will only continue to accel¬ 
erate as we get more frameworks, data 
and interest. Machine Learning sys¬ 
tems will become more adaptable in 
real time, and hardware acceleration 
will provide faster training and the 
ability to process more data. The im¬ 
plementation of machine learning in 
the modern world will be ubiquitous. 
Gars will be self-driving. Machines 
will support medical personnel, from 
assisting the elderly to supporting 
surgeons. Supply chains and financial 
markets will benefit from the analysis 
of millions of data points. Glassifica¬ 
tion and generation will be applied 
to the arts. Gestures, images, sounds, 
text, and concepts will be combined 
through machine imagination to pro¬ 
duce desired new artifacts or to search 
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for existing creations. New artifacts -- 
visual, literary and musical — will also 
be generated entirely by machine. 

Sebastian Raschka | Ph.D. candidate 
at Michigan State University, data 
analyst, book author, python pro¬ 
grammer, open source contributor 
and computational biologist 

Even people who aren’t machine 
learning researchers or practitioners 
have probably heard about deep learn¬ 
ing. If we consider deep learning as a 
specialized sub branch of deep learn¬ 
ing that is focussed on the training of 
deep artificial neural networks, then 
I’d say deep learning is probably the 
area that gets most of the attention 
from academic researchers, research¬ 
ers in industry, and practitioners, as 
well as popular media. However, that 
does not mean that non-deep learning 
machine learning is becoming unpop¬ 
ular or even redundant. It’s quite the 
opposite, not every problem domain 


with hundreds of thousands or mil¬ 
lions of training examples and there’s 
still a large demand for and interest in 
improving (and automating) “classic” 
machine learning. 

I see Artificial General Intelligence (or 
broad AI) useful as a goal to drive cur¬ 
rent research, but I don’t think that we 
will have systems with near-human 
intelligence in our lifetimes. An in¬ 
teresting direction, and maybe an im¬ 
portant component in future systems, 
is zero-shot learning to learn tasks 
without labeled data. Improvements 
in attention and memory are and will 
be another hot research topic. Further, 
I expect to see more research towards 
modeling 3D spaces (with a depth di¬ 
mension) instead of just flat images, 
which is especially interesting in the 
context of self-driving cars. Another 
potential focus of future research will 
be to protect models against exploits — 
looking at recent literature, it’s almost 
creepy how easily deep learning-based 


object recognizers can be fooled. 

AGISI I Independent research or¬ 
ganization dedicated to working on 
the development of scientific theor¬ 
ies and solutions towards the aware¬ 
ness and oversight of the transform¬ 
ative impact of AI 

Machine Learning systems are being 
used to help make decisions both large 
and small in almost all aspects of our 
lives, whether they involve simple 
tasks like recommendations for buy¬ 
ing books, email spam filtering, or 
purchasing insurance policies, to more 
objective matters like the prognosis 
of credit rating in loan approval de¬ 
cisions, health diagnosis, or the profil¬ 
ing of individuals by law enforcement 
agencies. They capture information 
on our daily activities, from exercise 
and GPS data that tracks our location 
in real time, to emailing and social 
media interests. They are increasingly 
used in our cars and our homes, in 
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semi-autonomous surveillance applications, for managing 
nuclear reactors and demand across electricity grids, and 
for boosting productivity in the business environment. 

Many uncertainties exist concerning future developments 
in the Machine Learning (ML) field. Although particular 
narrow Artificial Intelligence (AI) models already demon¬ 
strate aspects of intelligent abilities in limited domains, 
at this point they do not represent a unified model of in¬ 
telligence. There is much work to be done before true AI 
is “amongst us”. There are also many safety challenges to 
overcome such as security, data privacy and other tech¬ 
nological problems still requiring breakthroughs. At the 
present time it is difficult to predict the short term extent 
with which AI, and especially ML, will impact on social and 
economic institutions but in the long term it could have 
a major negative consequence the social and economic ef¬ 
fects of which could be severe for millions of people. 

■ FILIPPO LORENZIN 
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ARTIFICIAL INTELLIGENCE; 
MACHINE LEARNING^ M 
AND COMPUmjQNAL^ B 
CREATIVITY 




“The Post- Human era begins in full 

when the output of computers is unpredictable” 

The Post Human Manifesto 8.1 


by Claire Burke 
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CHAPTER 1 

MACHINE INTELLIGENCE 

Ever since the Industrial Revolution, 
machines have provided artists with 
new mediums to engage with. In our 
current cultural state, art and techno¬ 
logy are more intertwined than ever 
before. The word technology has it 
origins from the Greek word tekche 
which means art and logia which 
means the study of a craft or art. Tech¬ 
nological determinism is shaping the 
way in which art is evolving and being 
created, distributed, preserved and ex¬ 
hibited online. 

The relationship between human’s 
and machines is a complex one, it is 
important to understand the ways 
in which cognitive technologies and 
humans interact. “The relationship 
between artist and computer can be 


symbiotic for each depends on the 
other, and both do together what 
neither could do alone”. (Mallary, 
1979). 

Bohm suggests that understanding 
the operations of the mind is by dis¬ 
covering the distinction between in¬ 
telligence and thought. (Bohm and 
Nichol, 2000) 

Artificial Intelligence is the concept of 
making a machine that has the ability 
to think. Although machines can learn 
and solve problems, the inherent dif¬ 
ference is that of our human under¬ 
standing that machine cannot think. 
But is there really a difference between 
human and machine thinking? 

In 1872 the worlds first programmable 
computer The Analytical Engine was 
invented by Babbage and Lovelace. 


The analytical engine could preform 
computational tasks that would re¬ 
quire expanded human knowledge, 
they were not considered to have the 
ability to think. In certain areas such 
a playing chess and computation ma¬ 
chines have proven to surpass human 
intelligence. What can we expect for 
the future implications of art, when 
it is possible for machines to have the 
intelligence and ability to provide us 
with new and possibly better forms of 
art? 

1.1 TURING 

The last considered Artificial Intel¬ 
ligence relevant research was during 
the Dartmouth Summer Research 
Project in 1956. Alan Turing, the 
British code breaker and computer pi¬ 
oneer demonstrated the human mind 
from a computational point of view. 
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He demonstrated that human being’s 
have a capacity to build things but with 
regard to his research on computer 
machines Turing states his query “I 
propose to consider the question, “Can 
machines think?” This should begin 
with definitions of the meaning of the 
terms “machine” and “think.”(Turing, 
1950, p.433). Central to this argument 
we need to look at what is a mind and 
what is a machine. Turing devised a 
test to explore intelligence to study 
how machines respond. The test in¬ 
volves interaction between a human 
and a machine. A blind test consisting 
of several specified actions the human 
is asked whether it was communicat¬ 
ing with a human or a machine. If the 
human thinks this interaction with 
the machine was with a human, then 
the machine has been successful and 
subsequently passed the Turing test. 

The concept of intelligence has played 
a role in the way in which we under¬ 
stand ourselves. Nath acknowledges 
the relevance of Turing’s work “The 
Turing test was designed to provide 
us with a definition of intelligence”. 
(Nath, 2009). 

His analysis involved undertaking 
The Imitation Game. This was con¬ 
ducted by two people and one ma¬ 
chine playing the game, with one of 
the persons acting as interrogator. By 
asking questions of both the machine 
and the other person, the interrogator 
attempts to predict who is the ma¬ 
chine and who is the human. 

The field of artificial intelligence is 
relevant to research within art as it 
is uses logic, reasoning and decision 
making. Attempts to build machines 
with cognitive abilities began after 
the second world war. The invention 
of the first programmable computer 
in 1956 was the starting point for the 
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field of artificial intelligence research. 
(P, 2001) Theories began to emerge 
about the possibilities of creating a 
thinking machines as well as crit¬ 
ical discussions. In the 1950’s Turing 
thought it was possible that machines 
could think. Machines had proved to 
be successful in preforming complex 
cognitive tasks by solving problems 
and playing chess but unlike human’s 
they had not shown that they are cap¬ 
able of consciousness. Although it is 
only an assumption that only humans 
can have this trait. 

In many ways artificial intelligence is 
still dependent on the programmer to 
develop to the complexity of it so that 
it can deal with as many scenarios as 
possible. Writing a program to learn 
is possible but quite complex; artificial 
intelligence is pseudo at best. We in¬ 
creasingly rely on technology to per¬ 
form our daily tasks; these are in the 
form of algorithms. Artists rely on 
algorithms in the production and cre¬ 
ation of art, technology is incorpor¬ 
ated into many aspects of creativity. 

In 1991 Moravec predicted that even¬ 
tually artificial intelligence could re¬ 
place humans in every essential task 
including perception, motor abilities 
and superior reasoning. (Moravec, 
1991) He suggested that Artificial In¬ 
telligence has “successfully imitated 
aspects of rational thought and full 
evolution of robot intelligence would 
act on our behalf as literal minded 
slaves”. (Moravec, 1991) In the time 
of computational abundance are we 
entering into a moment where we 
cannot encounter art without some 
form of computation integrated into 
the it’s creation, production and dis¬ 
play. There are benefits to accepting 
the computer as a tool in the process 
of thinking within art making. Lopes 
suggests that artists can use computers 
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as cognitive tools as the artist can cre¬ 
ate with their brains to process in¬ 
formation and can assign computers 
cognitive tasks. (Lopes, 2010) But 
where does the line draw between 
artist and machine? 

The Turing test proposes the question 
“Is there a difference between the op¬ 
erational systems in the human mind 
and those working in the computer 
(Nash, 2009, p.xii) The central chal¬ 
lenge is to examine the relationship 
between human mind and machine 
mind and how this can inform our 
understanding of creativity. Turing’s 
famous question was; Can Machines 
Think? A different question resides 
under the contemporary conditions in 
which we now live. This central con¬ 
cern is; can machines actually create 
credible art 

CHAPTER 2 

ART CREATED BY MACHINES 

Contemporary art involves artists 
who collaborate with machines in the 
creation of art. If the computer can 
make decisions and produce outcomes 
that an artist cannot anticipate, what 
does that mean for the future of art. 
Will a machine be able show true cre¬ 
ativity and could it be considered art? 

Contemporary artists are engaged by 
developments within artificial intelli¬ 
gence, they seek to find new forms of 
responses and artistic representations. 
Machines have proven themselves of 
being capable at outperforming their 
creators and have displaced them in 
modes of labour production. So is it 
possible that they could produce better 
art than humans? If a program is writ¬ 
ten and exists as a set of instructions 
to the machine it can be considered 
to operate in an algorithmic way. Art 
created through this type of method is 
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placing the human to influencing the 
work in a direct way. A program that 
is written with rules, allows then for 
the constraints of letting go of human 
decision, it is then up to the machine 
to output what it decides. When de¬ 
ciding if machines can learn and make 
decisions it is important to consider 
the type of algorithm that is operating. 
If it is learning with rules and output¬ 
ting its own decisions based on these 
rules, will the intelligence that has 
been created exceed the intelligence of 
it’s creator 

Most art critics along with many 
viewers of art are of the opinion that 
artistic merit is solely a human skill 
and a human experience. It is widely 
accepted that the human expression 
of creativity is what distinguishes us 
from machines. Many do not accept 
creations produced by artificial in¬ 
telligence as credible art. How do we 
justify the machine solely as an artist? 
If we perceive that machines cannot 
think or feel, how can we say that they 
are creators of art and not just cre¬ 
ations themselves. Is it not art until we 
treat it as art? 


With no definition of creativity, it is 
impossible to say that a machine is not 
creative. One of the functions of art is 
to provoke a perception. Bohm asserts 
that in almost any field creativity is 
possible and is founded on the percep¬ 
tion of what is new an also different 
from previous knowledge. (Bohm and 
Nichol, 2000). 

Feelings and emotional responses 
emerge when we perceive an art 
work. So if it is the art work itself that 
provokes this and is not necessarily 
directly from the artist or creator but 
the artwork produced, in this sense a 
machine is creative. 

2.1 HAROLD COHEN AARON 

Harold Cohen a painter and professor 
at the University of San Diego, de¬ 
veloped a computer program to create 
art. It is called AARON and is one of 
the first and perhaps one of the most 
famous creative Artificial Intelligence 
that exists. AARON’s paintings feature 
in many museums including the Tate 
Gallery and Stedelijk Museum in Am¬ 
sterdam. His creator Cohen does not 
consider the idea that AARON creates 


art, but his invention does raise the 
question of whether a machine can 
become a creator of art? Is it Cohen 
that is the artist or is it AARON? Co¬ 
hen is the writer of the program but 
the machine is learning and executing 
the outcomes. The role the artist takes 
here is different to a traditional use of 
a machine to aid in the creation of art. 
Cohen resumes the role of a creator 
in the sense of an inventor, teaching 
the programme he created to think, 
setting out parameters and set of rules 
and circumstances. Is the process that 
is being carried out a creative act? Per¬ 
haps the paintings that are produced 
are just objects that are not neces¬ 
sarily creative but exist as creations. 
Suggesting that AARON imitates hu¬ 
man creativity very well by producing 
credible line drawings, Pepperell af¬ 
firms that it is just tool to inspire new 
work and that machines cannot be 
accredited to the work as they haven’t 
thought up the idea (Pepperell, 1995) 

AARON can produce paintings of 
anything it has learned. Even though 
it is a functional drawing machine 
producing art works in the style of 
colourful geometric patterns, Cohen 
says he would prefer not to call the 
machine creative. 

He admits that it actually knows “ very 
little about the world - it recognises 
the shape of people, potted plants, 
trees and simple objects such as boxes 
and tables”. (Wakefield 2015) The ab¬ 
stract paintings AARON produced led 
him to be accepted as reputable col¬ 
ourist by his owner. Upon seeing the 
results, it produced Cohen stated that 
“he would not have had the courage to 
choose the particular colours”. (Boden, 
2010, p.93) The machine has learned a 
certain style of painting, it had rem¬ 
nants of Cohen’s own signature style 
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Meeting on Gauguin’s Beach - 1988 - Photographer: Cohen, Harold 


of painting. It is arguable that by en¬ 
gaging with a computational device it 
changed his creative process. By cre¬ 
ating AARON Cohen suggest that he 
will have the unique opportunity to 
become the world’s first artist to have 
an exclusively posthumous exhibition 
consisting of completely new and ori¬ 
ginal works. (Kurzweil, 2000) What is 
so unique about the work is that 
AARON, due to its intensive data 
set, can produce original artworks, 
it doesn’t not mimic the style of any 
other artist that already exists The 
drawing it produces are original and 
they do not need the assistance of 
a human’s to produce these genu¬ 
ine pieces. In Plato’s Theory of Art, 
Lodge Affirms that, “It is clear that 
what makes an artist an artist is his 
technique, his mathematics, his logic, 
his scientific method”. (Lodge, 1953, 
p.94) In this sense AARON as a ma¬ 
chine possess all these attributes and 
executes them in his practice. 

2.2 PAUL THE ROBOT 

Paul the robot is another computa¬ 
tional device who produces artwork. 
The London based artist Patrick 
Tresset uses computational systems 
which introduce artistic behaviour to 
machines. Having reaching somewhat 
of a stumbling block in his career, he 
considered the fact that if he needed 
a machine to preform to create draw¬ 
ings, he would have to teach them his 
process. “This entailed studying per¬ 
ception, cognition, motor control” 
(Tresset, 2016). 

In 2016 as part of the exhibition SEE¬ 
ING, what are you looking at? 3RNP 
which stands for 3 Robots Named 
Paul was installed at the Science Gal¬ 
lery, Dublin. This installation at the 
Science Gallery draws portraits of 
humans and this artwork in partic¬ 


ular shows a collaboration between 
art and machines. The piece involves 
a traditional art sketching experience 
with role reversal within its execution 
“the human takes the sitter’s role and 
is sketched by three robots”(Tresset, 
2016). The subject matter sits down 
for approximately 30 minutes, while 
three robotic arms, controlled by soft¬ 
ware draw a portrait. There are three 
cameras that observe and interpret the 
data of the subject from three differ¬ 
ent angles and distances. While the 
subject poses, a series of movements 
and mechanical noises occur. Aware 
of being depicted by a robot and an 


uncertainty to the outcome of its rep¬ 
resentational skills, it eventually pro¬ 
duces three different portraits. Each 
are slightly different in terms of com¬ 
position but they all are to a varying 
degree of likeness to the subject. 

Three unique and surprisingly life¬ 
like results emerge. But similar to 
AARON, they both have human cre¬ 
ators, so should it really be the human 
who writes the computer language 
that the work is accredited to, is it 
really computational creativity? Does 
the robot really create the artwork 
or is it the intention of the human 
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artist that created the hardware and 
software. There are also unexpected 
results, depending on the time of the 
day and the mood of the machine, 
the assistants in the gallery state that 
the machine can act temperamental. 
Referring to it as a having good days 
and bad days, it emulates the sense of 
a being with a presence in the gallery. 
Tresset admits that there are still ele¬ 
ments of surprise to the work that is 
produced as it is not something that 
he can predict. (Nosta, 2017) Are 
these unintentional glitches that are 
the failure of the machine, or are they 
intentional? 

2.3 THE NEXT REMBRANDT 

The Next Rembrandt, it is one of the 
most recent research experiments in 
the the field of computational creativ¬ 
ity. The work was created by teams 
from Dutch museums Mauritshuis 
and Rembranthuis. The project was 
realised through a collaboration with 
researchers from the Delft Univer¬ 
sity of Technology and supported by 
Microsoft. The idea was an attempt 
to create a new Rembrandt painting. 
It uses algorithms to create a new 
painting based on Rembrandts ori¬ 
ginal style of art work. It does this 
under certain conditions and data. It 
emulates his art using colour, com¬ 
position, style and depth of paint. The 
algorithm creates by analysing data 
which is based upon Rembrandt’s en¬ 
tire collection of art. “It uses data that 
includes information colour, dress, 
topic, demographics, composition 
and all the rest”. (Coldewey, 2016) 

The algorithm learns to distinguish 
the individual features of the sub¬ 
ject matter within his paintings. As 
his painting were usually portrait, it 
has data which includes many of the 


geometrical features of his work in¬ 
cluding angles and distances between 
facial features.” The portrait consists 
of over 148 million pixels, based on 
168,263 painting fragments from 
Rembrandt’s oeuvre” (Westall, 2016) 
The data used was 3d scans of over 
346 of his paintings. On analysis the 
algorithm learned to reduced it down 
to the most common factors in his 
work. The most typical subject matter 
was a “white male who was wearing 
black clothes”. (Reynolds, 2016) 

Using 3d printing it uses a height map 
and builds up printed layers and tex¬ 
tures to create of a work which could 
be mistakable for an original Rem¬ 
brandt. 

It is unique as it poses somewhat of 
a “controversial point between the 
worlds of technology and creativity”(- 
Nudd, 2010). It shows that computer 
generated art can be 
indistinguishable from 
human produced art 
and can also evoke the 
same feelings as an 
original work. Innov¬ 
ation Manager Em¬ 
manuel Flores states 
that the painting has 
already passed a kind 
of Turing test. He ad¬ 
mits that the institu¬ 


THIS DATA IS 
BASED ON WHAT 
IS EXPERIENCED 
AND A SYSTEM IS AT 
PLAY, SORTING THIS 
INTO A PARTICULAR 
CONFIGURATION OF 
PATTERNS,COLOUR, 
COMPOSITION 


tion doesn’t accept this is a Rembrandt 
and to some of the people involved in 
the process it is very confusing. (Gosh, 
2016) 

With any creation of art there is a 
composition made, based on data con¬ 
taining information. This data is based 
on what is experienced and a system 
is at play, sorting this into a particu¬ 
lar configuration of patterns, colour, 
composition etc. It is narrowed down 
and there are continuous variations 
until a realised art work emerges. 


Lodge defines that “ the activity of an 
artist is, in fact, a copying and imitat¬ 
ing, a varying rather than creating. So 
this would suggest that the Next Rem¬ 
brandt the machine is varying rather 
than creating but it is still an activity; 
one that is adopted by human artists. 

The machine takes in it’s data and in 
the case of the Next Rembrandt the 
results are a creation of an artwork 
that sit within a social context, open¬ 
ing up debated within the art world. 
The researchers have given the ma¬ 
chine a data base of art and it has cre¬ 
ated what they enabled it to do. 

CHAPTER 3 
MACHINE LEARNING 

Where as artificial intelligence aims 
to get machines to preform intelli¬ 
gent tasks, the intention of machine 
learning is to enhance 
knowledge. Deep 
learning is at the fore¬ 
front of computational 
progression and the 
algorithms and neural 
networks are behind 
some of the devices we 
interact with every¬ 
day day. The ability to 
learn evolved in hu- 
man’s beings and there 
is no evidence to suggest that it could 
not occur in machines. Using machine 
learning we may be able to finally dis¬ 
tinguish between a machine that im¬ 
itates creativity and one that could be 
considered truly creative. 

Significant advances have been made 
in the field of neural networks are a 
type of machine learning algorithm 
that uses neural networks to learn. To 
emulate the operation of the human 
brain individual neurons are organised 
into complex networks from which 
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The Next Rembrandt [2016) 3D printed - Available at: https://techcrunch.eom/2016/04/06/tech-imitates-art-in-a-3d-printe- 


machines can learn from experiences 
(Pepperell, 1995) They can be applied 
to creative applications and artists are 
engaging with these concepts. Being 
creative in art is comparable to byte 
manipulation in large numbers with 
complex algorithms. At a low level the 
execution of creation would run fast. 
A high level language like Java per¬ 
haps could work with an algorithm, 
like an electronic canvas with each 
point having an x, y coordinate you 
can create shapes by changing col¬ 
ours within a given set or coordinates. 
These autonomous machines have 
skills; they are painting portraits that 
articulate technical composition sim¬ 
ilar to humans. 

Artificial neural networks are being 
trained to learn, they are showing 
multiple examples of what they should 
learn, they learn what data to ignore. 
The origins are algorithms that try to 
mimic the brain, they are responsible 
for the revival of machine learning 


due to the biological application. Ma¬ 
chines need data to learn, much like 
humans learn from data existing in 
the real world. There are many ways 
in which an image can be manifes¬ 
ted, but before this happens neural 
networks must learn their subject 
matter. Like human mind, the more 
information it has the more intelli¬ 
gent it can become and it learns from 
previous experience. The brain is the 
largest and most impressively func¬ 
tioning machine, so neural networks 
were based on algorithms that tried 
to mimic the brain. The neural nets 
are trained and it learns the connec¬ 
tions between its inputs and outputs. 
The mind is able to solve problems, 
learn new information and disregard 
previous collected and stored inform¬ 
ation and create new forms of insight. 
If computers now have the power, 
speed, and capability to process so 
much data that this type neural based 
algorithms have, they may posses this 
type of insight. 


3.1 ALGORITHMS 

It appears with regard to painting 
based machine learning, due to ad¬ 
vances within image classification, 
machines can output and create like 
humans. But will machines ever have 
intuition and an ability to think? 

Thomas Bayes an English mathem¬ 
atician who formulated a version on 
Bayes Theorem. “Bayesian statist¬ 
ics provide a framework for build¬ 
ing intelligent learning systems” 
(Ghahramani, 2014). Bayes Theorem 
is the concept that takes decisions 
based on the the probability of an out¬ 
come to happen, based on previous 
experiences. “Bayesian interference 
treats model parameters as random 
variables rather than as constants. 
The framework allows for prior in¬ 
formation to be formally taken in to 
account” (Puza, 2015, p.l) 

If this is how machines learn, it draws 


DIGIMAG the digicult's project journal I n° 76 I year XI I summer 2017 


25 




parallels with the concept of Dewey’s 
experience of art and the aesthetic. So 
it could be implied that if this is how 
humans critique and produce art then 
it could be possible to lead machines 
thinking creatively. So perhaps there 
could be thought to machine logic 
through inductive reasoning which is 
based on observation and experiment. 
A set of rules can take what has been 
useful and predict if the conditions 
will be useful in the future. 

A machines binary system, a true 
false logic is comparable to mod¬ 
ern philosophical logic. A possible 
route through gaining insight into 
how a machine may become creative 
is through neural networks because 
these are based on the human brain. 
Neurons function in a similar way to 
Boolean Logic, a true or false decisions 
using 0 and 1 in the form of AND, OR, 
and NOT. 

Artificial Intelligence did not gain 
traction after its invention and hit a 
stumbling block. This was mostly due 
to the technology not being power¬ 
ful enough to produce enough data. 
Neural networks had limitations due 
to the complexities of the systems. 
Moravec suggests that in the 1980’s it 
re-emerged within the field of cyber¬ 
netics under neural nets, this was due 
to the computer starting to become 
powerful enough to simulate interest¬ 
ing assemblies of neurons. (Moravec, 
2000) But still in 1995 Pepperell pre¬ 
dicted that machines would be “in 
the near future creative aids rather 
than creative tools”. (Pepperell, 1995, 
P.138) 

In the year 2000 Kurzweil predicts 
that by the year 2020 computers will 
achieve the capacity and speed of 
the human brain. He suggests that 
to achieve human intelligence that 
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one possible approach would involve 
copying the brain neural circuitry. 
(Kurzweil, 2000) Neural networks 
do present the evidence that ma¬ 
chines have the intelligence to learn. 
Through engagement with neural 
networks that learn from data, we can 
possibly gain greater insight into the 
possibility of intelligence and its pur¬ 
pose within machine creativity. 

CONCLUSION 

Human creativity is hard to define; 
in a bid to understand it perhaps in¬ 
stead of seeing artificial intelligence 
as a negative impact, we can look to 
redefine what creativity means within 
the contemporary. Turing realised 
the similarity between the process of 
human thinking and computational 
thinking in the 1950’s, the investig¬ 
ation of how computational devices 
create and think has only begun to be 
explored. Ascott suggested that there 
is a need for artists and technologists 
to collaborate to create telematic net¬ 
works and nodes of digital hardware 
and cybernetic systems, that will sup¬ 
port new forms of art practice. (Ascott 
and Shanken, 2003) “Artificial Intelli¬ 
gence has successfully imitated the 
conscious surface of rational thought, 
and in doing so made evident the vast 
unplumbed sea of unconscious pro¬ 
cesses below” (Moravec, 2000. P.24) 
Kurzweil suggests the scenario of a 
future large neural net which is based 
on reverse engineering of the human 
brain. It has the ability to access hu¬ 
man knowledge and learn language, 
it can develop its own conception of 
reality. We need to think about how 
computational devices are not just 
tools anymore that aid in the creation 
of art, they may need to be considered 
on their own merits. Can 
humans and machine work together 
to preform creative collaborations? 
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Rather than the fear of robots produ¬ 
cing better art and eradicating con¬ 
cepts of human creativity, we should 
look at the importance of how ma¬ 
chines can help humans understand 
and create art. The Art and Artificial 
Intelligent Laboratory at Rutgers Uni¬ 
versity is at the forefront for research 
within the field of machine learning 
applied to art. It’s most recent re¬ 
search involved developing the uses of 
a computational system that takes the 
role of the artist and produces com¬ 
pletely new artworks. (Voon, 2017) 
The study involved participants being 
shown a mix of artworks that were 
generated by machines, famous hu¬ 
man abstract artists and artists that 
were showing work at Art Basel 2016. 
Tresset defers that there is no such 
thing that exists as a fully autonom¬ 
ous system due to the fact that there 
is always a human creator or designer 
that has made decisions behind the 
machine. The machines behaviour is 
influenced by humans and therefore 
are a factor in the output it creates. 

Art practice that involves computer 
programmes involves exploring new 
technological implications. There 
a set of conditions and parameters 
where there is an output. Criterions 
series is practical research project 
that culminated in the realisation of 
two art works. Using Processing a 
visual computing program, computer 
code is running on a machine and 
captured frame by frame to result in 
a video piece. The code is a creative 
instrument of expression. The artist 
has control of the aesthetic using the 
movement. 

Pointing to the fact that these types 
of artworks are importantly pro¬ 
grammed by humans, Pepperell de¬ 
scribes the process as “long strings 
of commands that, even running on 
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the fastest machine, can only 'think’ 
one thing at a time” (Pepperell, 1995, 
p.l27) He asserts that to write a pro¬ 
gramme that mimics the workings of 
the brain you would need more time 
than existence itself. (Pepperell, 1995) 
That may have been true at the time, 
but over 20 years later with techno¬ 
logy’s predicted rate of evolution, we 
are beginning to considered that in¬ 
telligent machines may be a reality 
before we expected. Art created in 
this manner is a collaboration with a 
machine, it does require a human to 
program to set up various conditions 
and parameters. 

In the case of referring to art that in¬ 
cludes neural networks, the human 
is eventually relinquishing control as 
once the machine has learned based 
on the data provided, humans are no 
longer involved, they have just made 
the process possible. Machine learn¬ 
ing and algorithms currently aid hu¬ 
man creativity and a possible way to 
for a machine to show true creativity 
is by the use of neural networks. But 
does it matter if there is a human cre¬ 
ator who starts the creative process, 
when machines have shown them¬ 
selves once given the starting skillset, 
can learn and go beyond the ability of 
the creator? The results of the Rutgers 
study showed that the participants 
had an aesthetic preference to the 
machine generated art works. The 
machine whether considered creative 
or not, what it has learned in terms 
of pattern, colour and the choices it 
makes are to the human eye aesthet¬ 
ically pleasing. Humans cannot tell the 
difference between machine art or of 
that made by a human artist, perhaps 
it is because mostly the art works are 
new. They are unique and different 
and exist as a works in themselves, 
even if they are on some level copying 
a style. 

DIGIMAG the digicult's project journal I n 


As Walter Benjamin stated “In prin¬ 
ciple, the work of art has always been 
reproducible”. (Benjamin and Under¬ 
wood, 2008, p.3) In a sense machines 
are reproducing human artworks but 
arguably better, their masters are the 
computer programmers who feed 
them data consisting of various ar¬ 
rays of human made art from which 
they learn aesthetic judgement. The 
upcoming exhibition at the Museum 
of Modern Art, New York is titled 
Thinking Machines: Art and Design 
in the Computer Age 1959-1989 is a 
collection of work that will include 
artworks that were created using 
computers and computational creativ¬ 
ity. (Sterling, 2017) 

A retrospective that will “trace how 
computers transformed aesthetics 
and hierarchies, revealing how these 
thinking machines reshaped art”. 
(Sterling, 2017). 

Pepperell states that “Creativity, as 
we normally understand it, consists 
of acts of transformation that bring 
about something which is both new 
and beneficial. (Pepperell, 1995, 
p.ll5) Perhaps we must allow ma¬ 
chines a have a chance as artists, as 
the act of creativity may not need to 
represent anything other than the cre¬ 
ation of something new into existence 
that otherwise would not exist, 
mean using specific methods that are 
not necessarily “native” of a particu¬ 
lar discipline in order to address and 
understand a particular phenomenon 
better, or simply in a non-conven- 
tional way. But wouldn’t this anchor 
us to the very discipline from which 
we wanted to depart, and simultan¬ 
eously run the risk to de-legitimate 
our research effort? In either ways, 
what emerges is an inherent inabil¬ 
ity to “think” in terms of uncertainty 
(when understood as practice) as our 
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very mind is hardwired to concepts of 
boundaries, established conventions 
and definite outcomes. 

■ CLAIRE BURKE 
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AnalyticalMachine_Babbage_London - https://en.wikipedia. 0 rg/wiki/File:AnalyticalMachine_Babbage_London.jpg 
Babbage’s nalytical Engine, 1834-1871. (Trial model). Science Museum, London 


Lady Ada Lovelace (1815-1852), 
mathematician and theoretical com¬ 
puter programmer, was only 17 years 
old when she met Charles Babbage 
(1791-1871), who was 42 at the time \ 
cite{Fuegi2003}. Babbage was already 
known for designing complex mech¬ 
anical calculators, such as the famous 
Difference Engine (ca 1822); and he 
arguably invented the first digital, 
general purpose, programmable 
computer: The Analytical Engine (ca 
1837). Though he never actually built 
these machines due to funding issues, 
the designs remain influential to this 
day. 

Despite their age gap, Lovelace was 
Babbage’s intellectual collaborator and 
peer. It’s through her notes that we 
know so much about these machines, 
including what is considered to be the 
very first complex computer program. 
Her notes also include beautiful com¬ 
ments with incredible foresight, such 
as “The Analytical Engine weaves al¬ 


gebraic patterns just as the Jacquard 
loom weaves flowers and leaves”. 
— Ada Lovelace (1815-1852), “Notes 
(on The Analytical Engine)”, 1843 

She had even more profound insights. 
While Babbage was mostly interested 
in designing a machine that could cal¬ 
culate anything, i.e. operate on num¬ 
bers, Lovelace saw the potential of 
this machine to go beyond that, and 
operate on symbols, to do true general 
purpose computing. She went even 
beyond that to predict the generative, 
computational art movement to come 
a century later "... the engine might 
compose elaborate and scientific 
pieces of music of any degree of com¬ 
plexity or extent.” 

Also amongst her notes, is this very 
famous, oft-quoted, and somewhat 
controversial statement... 

“The Analytical Engine has no preten¬ 
sions whatever to originate anything. 


It can do whatever we know how 
to order it to perform. It can follow 
analysis, but it has no power of anti¬ 
cipating any analytical revelations or 
truths. Its province is to assist us in 
making available what we are already 
acquainted with.” 

Eor the last two centuries, researchers 
have been trying to prove her wrong. 
Researchers of so-called ‘Strong 
AL—those who are trying to build 
'truly intelligent’ machines; and re¬ 
searchers of so-called 'Computational 
Creativity’ —those who are trying to 
build 'truly creative’ machines. 'Intelli¬ 
gence’ and 'creativity’, two vague con¬ 
cepts that are often intertwined; and 
lack clear, universally accepted defin¬ 
itions. In fact this is indeed a bit of a 
problem in the field. 

Throughout the history of artificial 
intelligence, there has been a pattern. 
A problem or task is presented as the 
epitome of intelligence, as something 
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that only a human-level intelligence 
could accomplish. And if and only if 
a machine could perform that task, 
then it would be considered truly in¬ 
telligent. 

Once upon a time it was thought that 
simple arithmetic was something that 
only a human mind could grasp. That 
was proven wrong centuries ago with 
the advent of mechanical calculators. 
Then during the official birth of the 
field of Artificial Intelligence (Al) in 
the mid 1950s, mathematical proofs 
were set as the goal. Surely a machine 
could not perform the required logical 
reasoning to prove a mathematical 
theorem? Surely that was something 
only a human could do? 

By 1956, Allen Newell, Herbert A. Si¬ 
mon and Cliff Shaw’s ‘Artificial Intelli¬ 
gence’ “The Logic Theorist” was prov¬ 
ing theorems \cite{McCorduck2004}. 
Some people thought that the prob¬ 
lem of AI was solved, and within a few 
years we’d have full human-level in¬ 
telligence. Others quickly realised that 
this wasn’t really ‘intelligence’, it was 
just some code that some guys wrote, 
that randomly searched for a solu¬ 
tion, with some basic rules of thumb 
(“heuristics” as it was called) thrown 
in there. 

Chess, now that seemed like a true 
challenge. “If only a computer could 
beat a human expert at Chess”, it was 
proposed, “then it could be considered 
intelligent”. 

But in 1997 when IBM’s Deep Blue 
beat chess Grandmaster and world 
champion Garry Kasparov, many were 
quick to point out that Deep Blue still 
wasn’t really ‘intelligent’. It was just a 
really really fast computer. It was just 
brute force power trying millions of 
combinations of moves to see which 
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one worked best - again with a bit of 
heuristics \cite{Campbell2002}. That 
doesn’t count. Not intelligence. 

The game of ‘Go’ on the other hand, 
is different. It’s a game where having 
brute force power to search all the 
moves just isn’t enough. It was thought 
that ‘Go’ is a game that requires intu¬ 
ition, and a level of planning and gut 
feel combined with intelligence that 
only a human —or human-level intel¬ 
ligence—could have. 

But when Google DeepMind’s Al- 
phaGo beat world-ranking players 
Fan Hui in 2015, Lee Sedol in 2016 
and top ranked player Ke Jie in 2017, 
it still wasn’t satisfying to some. It still 
didn’t feel like ‘intelligence’. It just felt 
like some fancy statistics applied to a 
search tree, with some pattern recog¬ 
nition algorithms learning to predict 
optimal moves and estimate who’s 
leading just by looking at the state 
of the board \cite{Silver2016}. It still 
doesn’t count as ‘real intelligence’. And 
when some people refer to AlphaGo 
as ‘creative’, others are quick to shoot 
it down, citing sentiment similar to 
Lovelace’s. So there is a bit of a ‘mov- 
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ing goal posts’ problem in the field of 
AL Trying to pin down what exactly 
‘intelligence’ entails is very tricky. And 
this problem is articulated nicely by 
cognitive scientist Douglas Hofstadter 
\cite{Hofstadter 1979}: 

“Sometimes it seems as though each 
new step towards AI, rather than 
producing something which every¬ 
one agrees is real intelligence, merely 
reveals what real intelligence is 
not.”—Douglas R. Hofstadter, “Godel, 
Escher, Bach: An Eternal Golden 
Braid”, 1979 

In other words, if you know how it 
works, it ain’t intelligence. 

All sense of intelligence and creativity 
that might be attributed to a system 
seems to be destroyed once we under¬ 
stand the algorithm that computes the 
output. I wonder, if we ever do fig¬ 
ure out exactly how the human brain 
works, and the algorithms that drive 
human intelligence and creativity, will 
we cease to see ourselves as intelligent 
or creative as well? 

Of course I don’t know how likely 



deepbluG_kasparov - http://mashabLe.eom/2016/02/10/kasparov-dGep-blue/#DxJPjH034Eqi 

The final, decisive game in the rematch between Garry Kasparov and the IBM computer Deep Blue on May 11, 1997 

(STAN HONDA/AFP/GETTY IMAGES) 
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debates, and dead-ends, we’re still ar¬ 
guing over this question. 

In his essay, Turing refers to 
Lovelace’s claim that 'The Analytical 
Engine has no pretensions to origin¬ 
ate anything’, as “Lady Lovelace’s Ob¬ 
jection”. He proposes that in order to 
be considered to 'originate’ anything, 
a machine should be able to surprise 
people, even its own programmer. He 
re-frames the question in the context 
of machines that can learn. Because 
two years prior, in a report called “In¬ 
telligent Machinery”, he had already 
theorised exactly that, something that 
neither Babbage nor Lovelace had 
foreseen: machines that can learn \ 
cite{Turingl948}. He called these 'un¬ 
organised machines’, loosely inspired 
by the neurons in the brain. And he 
had the foresight to propose: 

“Instead of trying to produce a pro¬ 
gramme to simulate the adult mind, 
why not rather try to produce one 
which simulates the child’s? If this 
were then subjected to an appropriate 
course of education, one would obtain 
the adult brain.”. 

He went on to add: 

“Machines take me by surprise with 
great frequency... An important fea¬ 
ture of a learning machine is that its 
teacher will often be very largely ig¬ 
norant of quite what is going on in¬ 
side.” 

And this is precisely where both 
the greatest strengths, and greatest 
dangers of machine learning al¬ 
gorithms lie. This is the greatest 
strength of machine learning, because 
this is how machines can potentially 
help propel us to new places, to help us 
see things that we otherwise wouldn’t 
be able to see. 
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that is, because... “If the human brain 
were so simple that we could under¬ 
stand it, we would be so simple, that 
we couldn’t.”—Emerson M. Pugh (c. 
1938). from George E. Pugh (1977) 
“The Biological Origin of Human Val¬ 
ues”. \cite{Pughl977} 

But I’d like to return to Lovelace, and 
her controversial statement... 

“The Analytical Engine has no preten¬ 
sions whatever to originate anything. 
It can do whatever we know how 


to order it to perform. It can follow 
analysis, but it has no power of anti¬ 
cipating any analytical revelations or 
truths. Its province is to assist us in 
making available what we are already 
acquainted with.” 

In his seminal 1950 essay, “Comput¬ 
ing machinery and intelligence”, Alan 
Turing addresses this statement \ 
cite{Turingl950}. He starts the es¬ 
say with the question “Can machines 
think?”. Seven decades later, after 
hundreds if not thousands of papers. 


Lee SedoL Alphago - https://qz.com/639952/googles-ai-won-the-game-go-by-defying-miUennia-of-basic-human-instinct/ 
Lee SedoL vs Google DeepMind’s ALphaGo 
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Self-Driving-Uber - https://www.uber.com/blog/pittsburgh/pittsburgh-self-driving-uber/ - Uber is amongst the many tech companies Looking to roll out self-driving cars. 


While it was theorised almost 50 
years prior, it was partly thanks to an 
immense amount of number crunch¬ 
ing and computational analysis that 
the human-machine combo teams at 
CERN were able to find the tiny per¬ 
turbation at around 125 GeV, amongst 
petabytes of data, to confirm the Higgs 
Boson \cite{Taylor2012}. 

Likewise, gravitational waves were 
predicted over a century ago, but only 
recently observed through recog¬ 
nising and isolating a chirp lasting a 
fraction of a second, amongst years 
of deafening background noise, to 
identify the remnants of minute fluc¬ 
tuations in the fabric of space-time 
itself, emitted from two supermassive 
black holes colliding over a billion 
years ago \cite{Svitil2016}. 

As more and more of our hard sci¬ 
ences (and increasingly, soft sciences) 
become computational, the impact 


of data science and machine learning 
on all of these fields will continue to 
radically grow. As a result, in the near 
future we can expect that machine 
learning will bring much more dra¬ 
matic developments across all of these 
domains. Perhaps helping us answer 
open questions in physics, or finding 
cures for diseases such as leukaemia or 
Alzheimer’s. 

However, there is of course a lot to 
be concerned about as well. The ex¬ 
istential threat of 'killer robots’ fuel 
the nightmares of some - mainly very 
privileged \cite{Crawford2016} - 
folks. And while the idea of'SkyNet / 
Terminator / Matrix’ style robots be¬ 
coming 'self-aware’ and trying to en¬ 
slave humanity is closer to the 'ridicu¬ 
lous’ end of the spectrum, the idea of 
(increasingly) autonomous weapons 
and algorithmic warfare is indeed a 
very real threat \cite{Musk2017}. 
There are other, potentially more 


immediate threats. Such as mass job 
displacements, as more and more ro¬ 
botic and algorithmic labour is intro¬ 
duced \cite{Tufekci2015}. Critics of 
these threats are quick to point out 
that where one job is taken by a ro¬ 
bot or machine, many more open up 
in the design, development, mainten¬ 
ance, management and deployment of 
these machines. However, these are 
of course very different skill-sets, and 
it may take decades - if not - much 
longer, for society to adjust. And more 
often than not these new roles and 
skill-sets are suited to different parts 
of society than those whose jobs were 
displaced. 

A further discrimination takes effect 
when predictions made by these learn¬ 
ing algorithms are used for critical de¬ 
cision making, like deciding whether 
or not somebody should be given a 
loan, or parole \cite{Angwin2016}, a 
job etc.; or where to police. Since the 
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data that these algorithms are trained 
on come from our actions, our histor¬ 
ical records, they learn our societal bi¬ 
ases and ultimately reinforce cultural 
prejudices and further cripple oppor¬ 
tunities for those who are already at 
disadvantaged positions in society. 
What’s even worse, is when these de¬ 
cision making algorithms are painted 
as neutral’ and ‘unbiased’, or even ‘sci¬ 
ence’, and closed to scrutiny or debate. 
This is further amplified by the fact 
that these algorithmic decision mak¬ 
ing systems are often developed for 
profit with proprietary closed-source, 
closed-data algorithms. 

BUT THERE’S MORE. 

Already today we can see a huge im¬ 
pact of learning algorithms on per¬ 
sonal privacy. It’s already possible to 
detect a person’s heartbeat instantan¬ 
eously within a few milliseconds ac¬ 
curacy, by learning the perturbations 
in wifi signals bouncing off their body 
\cite{Zhao2016}. It’s already possible 
to infer - with very high accuracy - the 
web pages somebody has visited on an 
encrypted website, simply by learning 
the statistics of packet sizes and how 
they change over time, even if the 
packets themselves are encrypted \ 
cite{Miller2014}. It’s already possible 
to identify faces in photos from only 
a few pixels \cite{Hu2017}, or even 
if they’re wearing masks or glasses \ 
cite{Singh2017}. It’s even been sug¬ 
gested by some researchers that it’s 
possible to infer with great accuracy, 
whether or not a person is a crim¬ 
inal, just from a single photograph 
of their face \cite{Wu2016a}- in ef¬ 
fect computational physiognomy \ 
cite{Arcas2017}. While this last par¬ 
ticular research was quickly rejected - 
even ridiculed - by the wider research 
community, the former examples are 
generally accepted as viable and im- 
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portant fields of research, and are 
representative of the direction that 
machine learning research is moving 
in, even though they have arguably 
equally terrifying connotations for 
privacy and human rights. 

But in an age where all sciences are 
increasingly becoming more compu¬ 
tational and data-driven, ‘machines 
that learn’ open up the future to bey¬ 
ond fathomable. We should be pre¬ 
pared for the possibility of much more 
extreme futures, not only through 
incremental change to existing tech¬ 
nologies as mentioned above, but 
radical changes to what we cannot 
even imagine. Perhaps we’ll be ‘driv¬ 
ing’ around in ‘bio-vehicles’ fuelled by 
photosynthesis. Or perhaps we’ll ge¬ 
netically engineer ourselves to photo- 
synthesise. No more need for food, or 
steak. Sunbathing could not only give 
us all the energy (and stabilisation of 
entropy) that we need, but we could 
maybe even engineer it so that it tastes 
good, better than the finest steak, but 
without the carbon footprint. In fact, 
since we’ll be photosynthesising, we’ll 
be converting carbon dioxide into 
oxygen, and we could be directly off¬ 
setting the carbon footprint of our 
other industries! 

Of course I don’t actually believe this 
will happen. One doesn’t need to have 
a lot of foresight to realise how ridicu¬ 
lous this is. Both in terms of scientific 
plausibility, and in it’s optimism. But 
I will not be surprised, if attempts are 
made in this direction some day. Per¬ 
haps a slightly more plausible idea, 
is that maybe one day in the mid-to- 
near future, we’ll figure out how to 
genetically modify ourselves to boost 
our intelligence. Perhaps a medical 
procedure carried out in the early days 
of pregnancy can catapult an embryo’s 
potential future cognitive capacity 
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to many-times that of the smartest 
people to have ever lived. Imagine the 
state of the world then, when designer 
parents can boost the IQ of their de¬ 
signer babies to well over 300 (if IQ 
still is a valid metric by then). Also 
imagine, that this procedure is tightly 
controlled, and incredibly expensive, 
available only to an elite super-rich. 

I’m not for one second proposing that 
any of these particular scenarios are 
likely to happen. They may very well 
defy the laws of physics and biology, 
and no amount of technology can 
change that. But I have no doubt, that 
similar technologies and discoveries 
that sound ‘outrageous’ or ‘ridiculous’ 
to us now, will become a reality in the 
near future. These will be technolo¬ 
gies and discoveries that sound as ri¬ 
diculous to us now, as it was ridicu¬ 
lous to claim to someone in the 19th 
century that one day humans would 
be walking on the moon; or that we 
would send a flying robot through 
space for 6.4billion km, catapulting it¬ 
self around planets for over 10 years, 
to land on a tiny rock hurtling through 
space at 135,000 km/h. 

We have no idea what the limits of 
what machine learning can achieve. 
But we should be prepared for such 
- or arguably more - unexpected and 
radical developments in the mid-to- 
near future. And whatever techno¬ 
logical breakthroughs await us, it’s 
safe to assume that machine learning 
will play a significant role in at least 
some, if not most of them. While it’s 
difficult to predict right now exactly 
what is possible and what the future 
holds, a useful rule of thumb is to re¬ 
member that: If any kind of meaning¬ 
ful information is somehow encoded 
and hidden in a large amount of data, 
it’s plausible to consider that one day, 
with the aid of machine learning, we 
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may be able to extract that informa¬ 
tion. And while it’s tempting to think 
that this is always a good thing, it may 
not necessarily always be to the ad¬ 
vantage of the greater population or 
planet. 

The most important question 
amongst all of this is: the people who 
are designing, developing and deploy¬ 
ing these technologies, are they able 
to foresee the wider social impact of 
their decisions and actions? Or even 
if they can foresee the impact, is it 
aligned with your best interests? Are 
the decisions they’re making in line 
with a direction that you approve of 
or desire? What kind of values do 
these people have? Do they repres¬ 
ent you and your well being? This is 
why it’s absolutely essential that these 
teams have the diversity to represent 
as wide as possible, a range of both 
professional and personal experiences, 
perspectives, values, knowledge and 
opinions. Everybody’s voice is cru¬ 
cial in steering this, so that progress is 
made, not only in directions that be¬ 
nefit only a few (often at a huge cost to 
others) while propagating a culture of 
compliance\cite{Franklin 1989}; but 
in directions that benefit us all. 
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Workshop at OpenOot Lab - Highlights 


MACHINE LEARNING 
FOR ARTISTS 


Lessons from one year of workshops 


by Gene Kogan 


Over the past year, I’ve been teaching 
a workshop I’ve usually called “ma¬ 
chine learning for artists” which is 
also the name of a free, online book 
I’m building with collaborators, con¬ 
taining supporting materials and 
educational resources about a subject 
I’ve been interested in for a long time 
which has received a groundswell of 
public interest in the last two years. 
I’ve been very lucky to be able to 


turn this into nearly a full-time job. I 
didn’t intentionally set out to do that, 
but each workshop seemed to bring 
about another invitation for one 
elsewhere. Over the past 14 months, 
29 workshops have been organized, 
taking place collectively over 76 days 
and roughly 500 hours in session, not 
counting the hours of spontaneous 
hacking which would occasionally 
follow. Many of them have been co¬ 


taught with friends and collaborat¬ 
ors [1] [2] [3] [4] [5][6], steering into 
numerous subtopics and application 
areas. Three were recorded and pub¬ 
lished online, to go with the screen- 
casts I posted from a class of the same 
name I taught at the ITP program at 
NYU last year, the first of its kind that 
I did. 

Workshops have varied considerably 
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in duration and location. In one or¬ 
ganized with School of Machines in 
Berlin, we met every day for a month, 
whereas others were scheduled over 
just a few hours in a day. They’ve 
taken place at universities, companies, 
makerspaces, conferences, festivals, 
and occasionally even outdoors (when 
the weather permitted). 

Some were decidedly high-level, fo¬ 
cused on conveying the theoretical 
and mathematical foundations of the 
research to a more general audience, 
or examining the social dilemmas 
posed by our increasing integration of 
these technologies into everyday life. 
But most workshops have emphasized 
making and hacking, appropriating 
these methods into the areas that par¬ 
ticipants care about, and encouraging 
a more active and personal engage¬ 
ment with the tools themselves. From 
design to architecture to music and 
many others, the applications within 
creative domains are numerous, di¬ 
verse, and growing continuously. 

This emphasis on hands-on experi¬ 
mentation has led to a virtuous circle, 
whereby workshops have fed back 
into the development of ml4a’s tools, 
culminating in a recent beta release 
of nearly 30 standalone applications 
which handle common machine learn¬ 
ing tasks. These generic tools help 
enable students to build prototypes 
much more rapidly, even if they have 
little or no prior coding experience. 
I’m always amazed to see what people 
of varying backgrounds, interests, and 
skillsets cook up with them, apply¬ 
ing them in ways I would have never 
dreamed up myself. Last month during 
a workshop at Copenhagen Institute 
for Interaction Design co-taught with 
Andreas Refsgaard, the students —in 
groups of three —managed to build 
from scratch a collection of mature 


projects that last year I wouldn’t have 
believed possible to finish in just a few 
days of practice. 

STRATEGIC UNPREPAREDNESS 

Early on, my workshops were planned 
carefully to be as prepared as possible 
to make the most use of the always- 
too-little time we had. As I got used 
to doing them, I began to loosen up, 
so as to let sessions go in the direc¬ 
tions that students favored, some¬ 
thing which neither they nor I could 
easily anticipate ahead of time. I began 
modularizing my teaching materi¬ 


als—both the presentation slides and 
software —into interconnected nodes, 
likening them to a graph through 
which we could find a path most 
suitable to their impulses, frequently 
leading into unforeseen territory. For 
example, demonstrations of computer 
vision software would unexpectedly 
provoke a spirited debate over ethics 
in machine learning, or a review of 
recent research papers would lead to 
speculation over potential art applic¬ 
ations. One of the most serendipitous 
instances of this occurred during a 
workshop at OpenDot Lab in Milan 
last November. Literally minutes be- 
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fore the second session began, I read 
news about the now-famed pix2pix 
paper and codebase by Phillip Isola 
et al, which had just been publicly re¬ 
leased the day before. I was so excited 
about it that I scrapped that morning’s 
plans and showed pix2pix to my stu¬ 
dents instead, sparking a brainstorm¬ 
ing session that would lead to the 
spontaneous creation of a collaborat¬ 
ive artwork we called Invisible Cities, 
in which we used pix2pix to generate 
satellite imagery and transfer the style 
of one city onto another. 

TALKS AS TINY WORKSHOPS 

Over the same timeframe. I’ve also 
given almost as many “talks,” by which 
I mean a lecture of no more than an 
hour, although with the shortest 
workshops being only 3-4 hours 
themselves, the distinction is not al¬ 
ways so clear. The 
lines are further 
blurred by running 
my talks from the 
same custom ap¬ 
plication I wrote 
to teach my work¬ 
shops with. Built 
on top of open- 
Frameworks along 
with some C-l-l- deep learning libraries 
[1] [2], some of my “slides” are actually 
self-contained real-time demos, in¬ 
teractively showing how to do things 
like identify objects in a camera feed 
and retrieve similar items, arrange 
and visualize large datasets of images 
and sounds, depict the internal states 
of convolutional networks, train your 
own to make music and play video 
games, and many others. By injecting 
my talks with fun demonstrations of 
these tools, I start to think of them 
as very short workshops in and of 
themselves, encouraging attendees to 
actively pursue a deeper study of the 


subject afterwards. 

ASPIRATION VS. QUALITY 
OF LIFE 

As a personal aside, the workshops 
have taken place in many different 
cities around the world, allowing me 
to travel to places I had never been to 
before, make many new friends, and 
meet in-person with people whom I 
had only known through the internet 
before. But they’ve not been without 
some drawbacks. They require a lot 
of preparation and continual main¬ 
tenance, taking time away from my 
own research and art practice, and 
have made it harder to pursue long¬ 
term commitments planted in one 
physical location. The lack of major 
institutional support means I am re¬ 
sponsible for most of the administrat¬ 
ive overhead as well, and I’m liable to 
losing entire afternoons 
just trying to keep track 
of my own invoices. The 
constant travel, although 
rewarding, can be drain¬ 
ing as well, and I usually 
make very little time for 
sightseeing. On one trip to 
teach a workshop at Par¬ 
sons New School in Paris, 
where I had never been before, I only 
managed to visit the Eiffel Tower in a 
mad scramble on my way back to the 
airport. 

SUSTAINABILITY AND EUTURE 
PROSPECTS 

The fees from the workshops have 
accounted for most of my income 
this year, allowing me to keep ml4a 
a free project—not only free in terms 
of costs, but free of obligations that 
could compromise its accessibility, 
openness, and respect for movements 
I support [1] [2]. This is the most chal¬ 


I’M LOOKING INTO 
WAYS OF EVOLVING 
THE FORMAT INTO 
SOMETHING WHICH 
CAN BE ADAPTED 
AND REPURPOSED BY 
ANYONE 
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lenging and frustrating aspect of the 
endeavor, and my stubbornness prob¬ 
ably costs me opportunities to expand 
its scope or reach new audiences. 
Not every potential contributor has 
as much privilege to slash and burn 
through in service of a free project as 
I do, and I probably spend too much 
of my time thinking about the next 
bit of content it needs, rather than its 
long-term sustainability. But as the 
book inches closer to a first draft. I’m 
beginning to consider crowdfund¬ 
ing platforms as a possible first step 
towards bridging this gap, and am 
actively seeking out feedback and ad¬ 
vice from people with experience in 
self-publishing. 

Additionally, I’m looking into ways of 
evolving the format into something 
which can be adapted and repurposed 
by anyone, and encouraging contribu¬ 
tions from more participants in order 
to facilitate the creation of a common 
set of baseline tools that are broadly 
useful for many people. This initiat¬ 
ive is inspired by the endless creativ¬ 
ity of my workshop participants, and 
technologists out there more gener¬ 
ally. There seems to be an ecosystem 
forming at the intersection of art and 
machine learning. The two discip¬ 
lines bring very different legacies and 
vocabularies to the table, yet they have 
much to offer each other. 

COMPLEMENTARY PARTNERS 

The great paradox of machine learn¬ 
ing for artists is its economics. In a 
field which is booming commercially 
and industrially, it can be challenging 
to make the case for spending time on 
being creative with technology in a 
way that has few immediate practical 
benefits, let alone for making “art for 
art’s sake.” Pursuing art as a career is 
difficult for most people; secure aca- 
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demic jobs require costly degrees, and 
the art world lacks the abundance of 
high-paying jobs that medicine and 
law can provide to justify similarly 
priced training. This lack of profes¬ 
sional infrastructure leads most engin¬ 
eers and researchers to forgo any such 
practice, fearing it won’t lead to viable 
employment, a reasonable concern. 
On the other hand, my own artistic 
engagement with machine learning 
has quietly given me real-world ex¬ 
perience in computer science, a field I 
never formally studied. More import¬ 
antly, it’s given me intangible skills 
that outlive the ephemeral year-to- 
year overturn of the technology itself. 
A creativity-driven approach engages 
people and helps communicate the 
topic to the general public. At the 
same time, the vitality of the machine 
learning sector within industry and 
commerce could potentially support 
outlets for people whose curious im¬ 
pulses are stifled by financial barriers. 
The mechanics of how a partnership 


between these two fields could work 
are still fuzzy to me, but the opportun¬ 
ity is impossible to ignore. 

TO LEARN SHALLOW LEARN¬ 
ING DEEPLY OR LEARN DEEP 
LEARNING SHALLOWLY? 

Deep learning poses some pedagogical 
challenges. First, the prerequisite soft¬ 
ware can be difficult to install, char¬ 
acterized by numerous and sometimes 
obscure dependencies, unpredictable 
runtime errors, and instructions tar¬ 
geted towards people assumed to have 
a background in computer science 
or software engineering. For those 
without one, debugging can be very 
time-consuming if it can’t be taken 
care of in advance, and distracts from 
the main educational objectives. 

Additionally, the software contains 
few of the high-level abstractions 
found in creative coding libraries. The 
algorithms are expertly hand-crafted 


to effectively do one narrowly-defined 
task, and do it very well. Thus, it can 
be difficult to apply the software cre¬ 
atively in ways that are much different 
from how the original authors already 
demonstrated them. Furthermore, the 
computational expensiveness of most 
deep neural networks makes virtually 
all real-time applications impossible. 
A lack of desirable and large enough 
datasets, memory restrictions, and 
various other complications reduce it 
further. 

Nevertheless, over the past two years, 
several developments have emerged 
which mitigate these difficulties. 
Software services like docker and 
paperspace make it possible to pre¬ 
pare and ship learning materials in 
self-contained modules, and make 
the needed hardware accessible for a 
reasonable fee. An explosion of open 
source sharing has further helped by 
opening up core implementations of 
known techniques in tensorflow and 
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Nabi workshop 

torch. This has greatly lessened the burden of overhead in 
setup, making it a practical component in many courses. 
Educational resources to get started with machine learning 
have become ever more abundant over the past two years. 
Online courses are plentiful [1][2] [3] [4] [5], as are com¬ 
pilations, visual guides, tutorials, and many others [1][2] 
[3]. Although they do not specifically target artists, they are 
general enough to be applicable across many disciplines. 
With these trends only accelerating. I’ve come to believe 
that it’s not only possible, but imperative to make machine 
learning and AI more democratized in technical, artistic, 
and creative work. The questions I’m already looking ahead 
to are how can this project sustain itself into the future? 
How can it be structured in such a way that participants 
are incentivized to contribute back to it? Can ml4a evolve 
toward something like a blueprint or a kit, which others 
can freely unpack to organize their own workshops in their 
own communities? If I’ve learned nothing else from this 
year, it’s that I can’t really anticipate what others can build 


from these materials. I still don’t fully know what this pro¬ 
ject should become, but I’m open to ideas. Let this reflec¬ 
tion be a first step towards finding out. 

■ GENE KOGAN 


GENE KOGAN 


GENE KOGAN IS AN ARTIST AND A PROGRAMMER WHO IS IN¬ 
TERESTED IN GENERATIVE SYSTEMS, ARTIEICIAL INTELLIGENCE, 
AND SOETWARE EOR CREATIVITY AND SELE-EXPRESSION. HE IS 
A COLLABORATOR WITHIN NUMEROUS OPEN-SOURCE SOFT¬ 
WARE PROJECTS, AND LEADS WORKSHOPS AND DEMONSTRA¬ 
TIONS ON TOPICS AT THE INTERSECTION OF CODE AND ART. 
GENE INITIATED AND CONTRIBUTES TO ML4A, A FREE BOOK 
ABOUT MACHINE LEARNING FOR ARTISTS, ACTIVISTS, AND CIT¬ 
IZEN SCIENTISTS. HE REGULARLY PUBLISHES VIDEO LECTURES, 
WRITINGS, AND TUTORIALS TO FACILITATE A GREATER PUBLIC 
UNDERSTANDING OF THE TOPIC. 
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SHARED 

ARTIFICIAL 

INTELLIGENCE 



by Alessandro Masserdotti 
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Toaster (addicted product - Simone Rebaudengo) 
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Keytweeter - Kyle Mcdonald 


In recent years, I have often had to 
reflect on the future development of 
interconnected objects and their im¬ 
plications and consequences at a so¬ 
cial, ethical and political level. 

The world of connected objects is rap¬ 
idly expanding and there will soon be a 
need for artificial intelligence capable 
of governing this multitude of objects. 

What follows is a brief summary 
that puts in line thoughts and reflec¬ 
tions arising from the first edition of 
the talk “loF - Internet of Furniture”, 
organized in the occasion of Milano 
Design Week 2017 by Opendot in col¬ 
laboration with the German interac¬ 
tion design studio IXDS. 

Nowadays we are accustomed and 
at our ease, sometimes even uncon¬ 
sciously, to share part of our lives 
online through everyday tools such 
as social networks and google maps, 
which trace our movements generat¬ 
ing a flow of data. This sharing rep¬ 
resents our way of living, moving, 
buying and preferring. We constantly 
leave a trace of what we do and where 
we are. This generates a fragmented 
identity that represents our digital life. 


My brief reflection on the subject 
starts with a provocative question 
that has to do with data, privacy and 
the consequences of sharing our lives 
online, inspired by the works of Kyle 
Mcdonald like keytweeter. 

What if everything were completely 
open? 

What could happen if everyone had 
access to everyone’s information? 
Literally everything, I mean user IDs, 
passwords, web histories, banking 
activities, emails, etc. that is to say, 
everything related to our digital iden¬ 
tity and not just what we consciously 
share on social networks. 

Could there be a benefit for the in¬ 
dividual and the community, rather 
than just for the companies that cur¬ 
rently hold these data? 

We would all have the potential to 
be “controlled”, we would all have 
access to everyone’s digital activities, 
we could all take possession of each 
other’s identity. 

It would indeed be a big problem, but 
if everything were really completely 


open, a malicious person would in 
turn be “seen and spied on”, and prob¬ 
ably more reluctance or a large num¬ 
ber of detractors who would report 
his actions as incorrect. 

I know, it is an anarchic-futuristic hy¬ 
pothesis but, at least on paper, I would 
not take it for granted that it would 
not work. 

In the IT field, we already have a very 
good example of how open source has 
been an innovation engine. The fact 
that many developers release their 
source code is an unprecedented in¬ 
novative boost. 

The Internet does exist and its exist¬ 
ence is 70% based on open and public 
source codes. 

Let’s try to apply this perspective to 
objects, connected, with open and 
shared data. 

Let’s give an example linked to the 
theme of loT and open source. 

“Brad” - The shared toaster is a 2012 
project by an Italian designer, Si¬ 
mone Rebaudengo (https://vimeo. 
com/41363473), who designed a 
toaster with a personality, intercon¬ 
nected and in constant dialogue with 
its peers. 

I think this project is instructive from 
many points of view, the first con¬ 
cerns the way we interface with Brad. 
We users use Brad as we have always 
done with all other toasters, without 
the need to learn a complex operation 
or having to enter custom parameters 
for toasting bread. But Brad is an in¬ 
terconnected object that communic¬ 
ates with other toasters. 

The objective of the project, ironic 
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and speculative, is the implication of the initiative, better said agency, by the 
objects, and their ability to perform smart actions regardless of our commands. 
The Brad toaster decides how to behave autonomously on the basis of the data 
he collects and shares with other toasters like it. 

The object in itself does not belong to me, but on the basis of data of use and 
exchange of information with other objects like it is able to move and move 
around the place where it is most needed. 

This is to say that loT products must, or should, be objects that do not interact 
directly with people, that do not require us to interface with them, and above all 
they don’t interface with us even when we do not want it! Today we are more 
than ever bombarded with notifications between mail, chat, social networks, 
and we don’t need more alerts. We do not need a refrigerator that warns us 



when tomatoes are missing, for ex¬ 
ample. Objects should have their own 
life, autonomous and detached, they 
should be able to interact with each 
other and not be designed for them to 
seem human, carrying out actions on 
our behalf. 

The real innovation would be to 
maintain a normal and fluid interac¬ 
tion between man and product, where 
the object is able to acquire its own 
autonomy to manage smart and auto¬ 
mated actions, regardless of the will of 
man. 

A practical example: we all have a box 
of medicines at home, how many of 
these packages have expired or are 
about to expire? We assume that our 
neighbour needs precisely that medi¬ 
cine that is about to expire on our 
shelf, a hypothetical “loT Medicine 
Box” could communicate with other 
“loT Medicine Boxes” in the neigh¬ 
borhood to indicate the availability of 
that medicine to those who need it at 
that precise moment. 

Another example, more immediate 
and increasingly topical: let’s think 
about what would happen if each city 
were equipped with self-guided cars 
capable of moving by themselves, 
which learn to move from one part of 
the city to the other at specific times 
according to the request and routines 
of the citizens. It would be a revolution 
on multiple scales as well as urban, 
environmental, social and economic 
one. The vision of the loT is towards 
a future in which objects are intercon¬ 
nected, able to dialogue and share data 
making them public and usable to all, 
for what we could call a Shared Artifi¬ 
cial Intelligence. Artificial intelligence 
should be thought not as a substitute, 
but rather as something complement¬ 
ary to human intelligence. 
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Museo Archivio del Diari, installation by Dotdotdot 


Even today, however, despite the not always satisfact¬ 
ory results, all research is trying to emulate the brain and 
human behaviour. Perhaps the question we should ask 
ourselves is: “What do computers and machines do better 
than us?” This is what research should focus on today: on 
the implementation of computer systems, on the devel¬ 
opment of machines and systems capable of establishing 
and maintaining multiple and synchronous relations with 
tools, objects, products, things, to control a large amount 
of data, rework them and compare them with others. The 
objects would thus acquire autonomy and an “own life”, op¬ 
timizing resources and helping us in our daily actions, but 
without interfering or changing our regular human habits. 

In short, it doesn’t matter if in the future we will have a 
computer capable of telling us good morning, it would be 
more useful to have a washing machine that is able to de¬ 
cide autonomously when to activate itself according to the 
energy availability of the electricity grid, perhaps agreeing 
with the other washing machines in the neighbourhood to 
decide the most efficient moment of the day to carry out 
the washing, beyond the will of its owners. 

■ ALESSANDRO MASSERDOTTI 


ALESSANDRO MASSERDOTTI 


ALESSANDRO MASSERDOTTI IS ONE OF THE EOUNDERS OE DOT¬ 
DOTDOT, A MILAN BASED STUDIO OE EXHIBITION AND INTER- 
AOTION DESIGN. HE STUDIED PHILOSOPHY AND SINCE 1996 HE 
WORKS AND PLAYS AROUND WITH PHOTOS AND VIDEO, IN 2002 
HE STARTED WORKING ON INTERAGTIVE AND SENSIBLE SPAGES, 
ESPEGIALLY MADE WITH VVVV, UNTIL RIGHT NOW. ALONG WITH 
HIS PROEESSIONAL JGB HE IS USED TO TEACH AT UNIVERSITIES 
AND AT DESIGN SCHOOLS IN MILAN AND ABROAD. HE LIKES TO 
RESEARCH AND EXPERIMENT, HE PARTICIPATES, AS DOTDOT¬ 
DOT, AT SHOWS AND EXHIBITIONS IN ITALY AND ABROAD. HE 
LOVES TO WORK IN TEAMS WITH MULTIDISCIPLINARY PEOPLE 
SHARING EXPERIENCE AND KNOWLEDGE 


DIGIMAG the digicult's project journal I n° 76 I year XI I summer 2017 


45 






"Counter” installed in Aarhus, Denmark (® 2011, G. Drake-Brockman) 

MIRRORBEINGS: 

ROBOT COMPLEXITY, 
MYTHS, AND SIMULACRA 


by Geoffrey Drake-Brockman 


This paper traces a 
speculative journey 
investigating the nature 
of "created beings” 
or "mirrorbeings” 

- machines that we 
make as reflections of 
ourselves. 


ROBOT FLOWERPOTS 

An origami “chatterbox” is a commonplace object. The chatterbox is familiar 
to us as a simple paper-folding project and as the basis for a popular children’s 
fortune-telling game. 1 became interested in the chatterbox because of its uni¬ 
versality and the intriguing lateral-shift spatial transformation it performs when 
operated with two hands and used in fortune-telling mode. By attaching this pa¬ 
per origami element to an electromechanical actuator 1 accidentally produced an 
alternate sudden inversion spatial transformation that is analogous to the trans¬ 
ition of a flower from bud to bloom. Further development from this starting 
point yielded a fully robotized flowerpot. This flowerpot has a mirrored upper 
plate through which emerges a cloth-covered telescopic stem. When activated, 
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the stem grows one metre vertically 
before the green origami bud atop 
it suddenly blooms into a pink and 
yellow origami bloom - whilst pro¬ 
ducing a distinct “wap” sound. Soon 
afterwards, the bloom withers back 
to its flowerpot and returns to a bud 
state. 

Floribots is an interactive collective 
organism consisting of 128 of these 
robot flowerpots with appropriate 
networking, electronics, sensors, and 
control software - it is a kind of ro¬ 
bot garden bed that combines the 
familiar and comfortable chatterbox 
motif with a “spooky” manifestation 
as a huge (8m x 4m) mechanical floral 
arrangement that “watches you” and 
constantly reconfigures itself. Con¬ 
ceptually, Floribots was intended to 
stage a real-world encounter between 
its audience and a kind of “sci fi” 
tableau of co-operating mechanical 
plants - while provoking in the ob¬ 
server hopefully equal measures of 
disquiet and attraction. 

When writing the software for the 
Floribots “hive mind” I drew on as¬ 
pects of the behavior of my then- 
toddler-aged children. Accordingly, 
Floribots was programmed to ex¬ 
hibit different “moods” including the 
following: Reactive, Excited, Bored, 
Naughty, and Sleepy. 

Floribots was first exhibited at the 
National Gallery of Australia in 2005 
where it interacted with an audience 
of some 100,000 visitors over a four- 
month period. In practice, the beha¬ 
vior exhibited by Floribots seemed 
to me much more complex than its 
predefined moods and the transitions 
between them that I had programmed. 
Sometimes mood behaviors effect¬ 
ively partially overlay each other. 


creating new choreographic modes, 
whist the sound compositions played 
by an orchestra of 128 “wapping” pa¬ 
per flowers were entirely novel. The 
interaction between the work and its 
audience proved to be intense. I ob¬ 
served people lingering near the work 
for long periods and found that they 
would refer to the actions of Floribots 
as though it was a “being”, rather than 
a mechanical arrangement of com¬ 
ponents. Floribots was voted “Peoples’ 
Choice” of the Na¬ 
tional Sculpture Prize 
in 2005. It was the first 
time that I had made 
a “popular” artwork - 
seemingly almost by 
accident. 

I understood the role of 
the commonplace ob¬ 
ject, the origami chatterbox, in mak¬ 
ing Floribots accessible to its audience 
- this was an intentional device. How¬ 
ever, I felt the engagement that the 
work engendered with its audience via 
its novel behavior-patterns required 
more explanation. I wondered if these 
behaviors could be understood in 
terms of “Complexity Theory” (Gleick 
1988). Complexity Theory investig¬ 
ates how relationships between parts 
of a system give rise to the collect¬ 
ive behavior of that system. A suffi¬ 
ciently complex system can sometime 
self-generate novel behaviors through 
a process called “emergence”. Along 
these lines, it seemed that the overall 
“phase space” defined by Floribots’ 
mechanical, electrical, and software 
freedoms had given rise to emergent 
patterns and expressions - effectively 
allowing a created being to come into 
existence. 

Perhaps, I thought, any sufficiently 
complex automaton has the capacity 


to become a being and exhibit novel 
behavior... 

To analyze Floribots in Complexity 
Theory terms, I assessed it logically 
- that is, as a state machine. A state 
machine is a device with a calculable 
number of discreet possible condi¬ 
tions. Floribots has 128 flowerpots, 
which can be independently switched 
between bloom and bud modes. Thus, 
Floribots can be regarded as having 2'^* 
(circa 10^*)potentially 
different states. Given 
its order of complexity 
of 10^* - correspond¬ 
ing to many trillion 
trillion trillion trillion 
states, I postulated that 
the shear extent of this 
complexity was the 
root cause of its novel 
(emergent) “being-like” behaviors and 
resultant intense levels of audience 
engagement. 

COUNTING 

After my experience with Floribots, 
I decided to create a new automatons 
artwork with much less inherent com¬ 
plexity - to see if emergent behavior 
still manifested. The work I developed 
was called simply “Counter” and was 
completed in 2009. 

Counter is an interactive installation 
in the form of a large yellow pedes¬ 
trian portal that literally counts each 
person that walks through it. Counter 
has nine magnetically-actuated di¬ 
gits on its front and back faces and is 
capable of counting to one less than a 
billion, after which it clocks-over and 
returns to zero. Each time Counter’s 
number changes; a distinct “thwack” 
sound is made as its magnetic display 
segments flip over. 


FLORIBOTS IS 
AN INTERACTIVE 
COLLECTIVE 
ORGANISM 
CONSISTING OF 128 
OF THESE ROBOT 
FLOWERPOINTS 
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Apart from its role in mapping the 
possibilities of mirrorbeings, the 
concept for Counter arises from the 
imperative to “be counted” or “make 
sure you count” that is part of liberal 
democratic cultural heritage. In addi¬ 
tion, the work carries more unsettling 
overtones of surveillance and scientific 
measurement. Counter exists to per¬ 
form a commonplace and straightfor¬ 
ward act; to count. It also employs an¬ 
other commonplace, highly accessible, 
motif - in terms of being, in terms of 
form, a simple “doorway”. 

Counter has 10^ potential states, mak¬ 
ing it hugely less complex than Flori- 
bots as a state machine. In addition, 
unlike Floribots - which can trans¬ 
ition between states in multivalent 
and open-ended ways - Counter has 
only one transition available: “to in¬ 
crement”. Despite being crippled 
in terms of its relative complexity. 
Counter has proven surprisingly cap¬ 
able of engaging its audience. The 
work has been installed four times in 
temporary outdoor exhibitions, three 
times in Australia and once in Den¬ 
mark. Each time the work has counted 
around 2 to 300,000 pedestrians, with 
its final installation at Bondi in Sydney 
taking it over one million aggregate 
interactions. 

Two arguably emergent behaviors 
have manifested repeatedly in all four 
of Counter’s installations; phenomena 
which I term the “pedestrian vortex” 
and the “decimal effect”. A pedes¬ 
trian vortex forms when a group of 
people form a circular queue to con¬ 
tinuously file through Counter’s arch¬ 
way and keep it “clocking over” as it 
counts each individual over and over 
again. It turns out that being counted 
repeatedly is sufficiently attractive 
for this formation to spontaneously 


Prototype robotic origami flowerpot [© 2005, G. Drake-Brockman) 

occur every time that Counter has 
been exhibited. The decimal effect is 
a heightened level of crowd engage¬ 
ment and excitement as Counter ap¬ 
proaches a large power-of-ten clock- 
over point, such as 10,000 or 100,000. 
At such moments some jostling to “be 
the one” occurs and a loud spontan¬ 
eous cheer will typically arise from the 
audience. 

I think that the most interesting, 
possibly emergent, behaviors of the 
Counter installation are not its mech¬ 
anical state changes in isolation, but 
the combination of these transitions 


with audience behaviors. Thus, the 
true complexity of the created being 
system includes not only the 10^ states 
inherent in Counter’s electronics but 
the much larger “phase space” of its 
300,000-per-exhibition human audi¬ 
ence. The ultimate complexity of such 
an automatons artwork becomes diffi¬ 
cult to fix, given the demonstrated ca¬ 
pacity of Counter to “grow” its phase 
space by absorbing state-potential 
from its human audience. 

Based on my experiment with 
Counter, I conjectured that even 
simple automata have the propensity 



DIGIMAG the digicult's project journal I n° 76 I year XI I summer 2017 


48 





"Floribots” responding to its audience [© 2005, G. Drake-Brockman) 


to “borrow” additional state-potential 
from their audience, so that they too 
can achieve emergence. 

BINARY AUTOMATA 

If an automatous artwork as simple as 
Counter can develop emergence, how 
simple can the system get, while main¬ 
taining this propensity? To invest¬ 
igate, I decided to address the logical 
limit of state machines. Accordingly, 
the next work in this series, titled 
“Clockwork Jayne” has just two states. 

Clockwork Jayne consists of a life-size 
fiberglass ballerina figure mounted 
on a faceted mirror base enclosing a 
clockwork mechanism that can rotate 
her. Clockwork Jayne was modelled 
on prima ballerina Jayne Smeulders 
of the West Australian Ballet, who 
heroically posed for over three hours 
standing “en pointe” while a full body- 
cast was made. When this work’s 
clockwork mechanism is wound up. 


the ballerina pivots slowly and a tune 
plays quietly until the spring winds 
down. The work draws on another 
commonplace motif; children’s clock¬ 
work music boxes with ballerinas that 
pop-up and spin in front of a mirror 
when you open the lid. As a simple ro¬ 
tating clockwork, this automaton has 
just two logical conditions: wound¬ 
up, and unwound. 

When Clockwork Jayne was exhib¬ 
ited, despite her extreme simplicity as 
a state machine, yet another self-gen¬ 
erating audience behavior was appar¬ 
ent: Clockwork Jayne would prompt 
her gallery audience to form into an 
orderly queue - a long line of people 
patiently waiting for the experience 
of winding her up and watching her 
gradually unwind. 

In observing this binary automaton, 
with a level of inherent complexity 
surely too low to permit emergence 
from within, 1 still noted an artwork/ 


audience interaction that was argu¬ 
ably emergent. My interpretation, 
consistent with my earlier conjecture, 
is that artworks are able to grow be- 
haviorally by acquiring state-potential 
from their human audience. 

The motivation for humans making 
their state-potential available to an 
automaton, however, requires further 
explanation. In the case of Counter 
the act of enumeration itself seems 
sufficient to prompt deep audience 
engagement with an abstract, concep¬ 
tual work. 1 see a parallel with the use 
of a commonplace motif in Floribots; 
just as flowerpots are familiar and at¬ 
tractive, so is the very act of counting. 
It seems that humans are generally 
attracted to automata based on com¬ 
monplace motifs and motivated to 
share state-potential with them. In the 
case of the ballerina automaton Clock¬ 
work Jayne however, my view is that 
it’s principally the device of anthro¬ 
pomorphism that binds the audience 
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"Counter" with a "pedestrian vortex" I® 2011, G. Drake-Brockmanl 


so closely to these human-shaped art¬ 
works. Humans are universally attrac¬ 
ted to representations of themselves, 
and the intensity of this reaction is 
magnified exponentially when the 
representation moves, and even more 
importantly reacts to them. With the 
boost provided by anthropomorph¬ 
ism, it seems that even binary auto¬ 
mata can achieve emergence. 

VARIABLE RELIEF 

Having explored the limit of low-com¬ 
plexity automata, an alternate wing of 
investigation suggested itself - that of 
automata even more complex than the 
2 '^* states of Floribots. Accordingly, 1 
created a “spatial robot” called “Head- 
space”. 

Headspace draws on the ancient art- 
form of relief sculpture, but updates 
the traditional carved stone format 
to a matrix of 256 motorized pol¬ 
ished aluminum rods. Each rod can 
independently move back and forth 
by about half a metre, allowing the 
overall grid to assume a wide range of 
relief topologies. Headspace is effect¬ 
ively a “variable relief’ sculpture. 


Headspace is fully autonomous; pos¬ 
sessing four motion sensors with 
which to detect human presence and 
an on-board software algorithm to 
regulate its behavior. This Headspace 
“mind” is loaded with three-dimen¬ 
sional scan data from the faces of 
over 600 schoolchildren, and the rod 
matrix is able to adjust its relief pro¬ 
file to represent these faces, as well 
as morph between them and perform 
various geometric transitions. As a 
variable-portrait system. Headspace is 
capable of human representation - like 
Clockwork Jayne - and thus is also, in 
a sense, anthropomorphic. 

Headspace is vastly more complex 
than Floribots. Each rod in the Head- 
space matrix may be moved between 
256 discreet positions, so the overall 
system has 256^^^ or about 10’’'^ states! 
When Headspace was placed on per¬ 
manent exhibition at Christ Church 
Grammar School (Perth, Australia), I 
noticed novel, unanticipated behavi¬ 
ors manifesting in terms of interfer¬ 
ence between its disparate subsystems, 
and apparent layering of algorith¬ 
mically discreet matrix behaviors. 
So Headspace also exhibits emergent 
behavior - as we may have expected 


given its very high level of complexity. 
In addition, Headpsace has proven to 
be exceptional in terms of its audience 
engagement, although mostly online. 
In fact, more than 160,000 people 
have viewed the Youtube video of 
Headspace - considerably more than 
have physically attended any of my 
gallery exhibitions. 

COSMIC CONNECTIONS 

Although Headspace is a permanent 
installation, it is located inside a build¬ 
ing and its physical audience is restric¬ 
ted to students and staff at one par¬ 
ticular school. 1 speculated that there 
could be other audiences and sources 
of state-potential available to a suit¬ 
ably optimized robotic artwork. The 
opportunity to investigate this possib¬ 
ility arose with a commission to create 
an external artwork for the NEXTDC 
Data Centre, in Malaga, Western 
Australia. The resulting work is titled 
“Readwrite” - and is activated by stim¬ 
uli of primarily extra-galactic origin. 

Readwrite is an autonomous robotic 
artwork some 10m in length, with 
24 pneumatically-actuated “flipping” 
elements arranged in a grid, moun¬ 
ted on the front elevation of the data 
center. Motion sequences on Read¬ 
write are triggered by the detection of 
charged “muon” particles. Muons are 
terrestrial cosmic rays generated in 
the upper atmosphere by interactions 
with high-energy particles originat¬ 
ing from distant supernovae and the 
accretion disks of supermassive black 
holes in active galactic nuclei. Read¬ 
write has four muon detectors - with 
one mounted at each corner of the art¬ 
work. When a “cosmic ray” hits one of 
the corners of the piece, a propagating 
wave of flipping elements begins from 
that point. 
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The Readwrite control algorithm is 
based on a heavily modified version 
of the Floribots code-base, and retains 
elements of the emotional modes of 
that work - which were originally 
modelled on the behavior of my sons 
at toddler-age. Given this, although 
Readwrite is lower in complexity at a 
mere or ~10^ states, it is not sur¬ 
prising that some of the propensity 
for emergent behavior first noted in 
Floribots remains evident. Indeed, 
Readwrite has been observed to per¬ 
form overlapping choreographies and 
mid-flip reversals which can be inter¬ 
preted as emergent behavior patterns. 
In terms of audience reaction. Read- 
write’s location - high on a building 
on an arterial road in an industrial 
precinct - means that little local im¬ 
pact is readily apparent - bar the oc¬ 
casional car slowing down to get a 
better view. Thus, it seems that Read¬ 
write is unlikely to be able to borrow 
much state-potential from its human 
audience. However, perhaps its ul¬ 
timate complexity as an automatons 
system extends to its network of ex¬ 
tra-galactic connections - which could 
give rise to considerable additional 
emergent potential. A caveat how¬ 
ever, is that the cosmic conversation 
in which Readwrite is involved is 
fundamentally reactive rather than 
interactive in character - due to large 
distances (millions of light years) ex¬ 
tending the feedback time from its ex¬ 
tra-galactic interlocutors beyond the 
likely endurance of the work. 

ROBOT MYTHOLOGIES 

I noted previously that the Glockwork 
Jayne, and Headspace automatons 
artworks employ the compositional 
device of anthropomorphism - that 
is, they mimic the physical appearance 
of a person. Extrapolation beyond the 
notion of anthropomorphism led me 


to wonder what it might be like for a 
created being to not just look, but to 
be, like a person - to delve into the 
realm of the “anthropo-onlological”. 
1 anticipated that even deeper levels 
of audience engagement should be 
possible with this approach, with yet 
greater potential for human phase 
space to be “shared” with an auto¬ 
maton. 1 decided that an investigation 
of this possibility would best be made 
via an ongoing dialogue 
between a real person and 
a made person - somewhat 
in the tradition of a Tur¬ 
ing (1950) Test. 1 have col¬ 
lected a set of pre-existing 
frameworks for such con¬ 
versations that I call “robot 
mythologies”. 

My list of candidate ro¬ 
bot mythologies includes 
widely known stories about made 
beings, such as: Mary Shelly’s (1818) 
Frankenstein - the creature who be¬ 
comes jealous of its creator; Pinocchio 
(Gollodi 1883) - the wooden boy who 
wants to be real; Rachel - the replic¬ 
ant who thinks she’s a real woman 
(Dick 1968); Terminator - the robot 
from the future that becomes a sur¬ 
rogate father figure (Gameron 1984); 
Golem - the clay being from Jewish 
mythology that is animated by an in¬ 
scription but cannot itself talk; the 
Tin Man - who yearns for a heart to 
fill his empty chest (Baum 1900); and 
the robot doppelganger of Maria who 
unleashes lust-driven chaos and stirs 
dissent throughout Metropolis (Lang 
1927). 

Perhaps the most emotionally-charged 
robot myth is Goppelia, as it deals spe¬ 
cifically with romantic love and attrac¬ 
tion. Goppelia is a story about a clock¬ 
work girl, who is mistaken for a real 
girl by a boy who falls in love with her. 


The story thickens further when the 
clockwork girl is in turn impersonated 
by a real girl, jealous of the boy’s affec¬ 
tions. Goppelia is a ballet, with music 
by Saint-Leon, Nuitter, and Delibes, 
based on a story by Hoffmann (1817). 
It was first performed in Paris in 
1870, and since then has become part 
of the classical ballet repertoire and is 
staged frequently by ballet companies 
around the world. Because the Gop¬ 
pelia story deals 
with issues at the 
edge of humanity 

- machines inter¬ 
changeable with 
persons, love and 
attraction in flux 
at this boundary 

- 1 decided it was 
fertile ground on 
which to develop 
an automatons 

artwork dealing with the crux of the 
created being issue. 

THE GOPPELIA PROJECT 

The Goppelia Project involves the 
creation of a troupe of four life-size 
autonomous robot ballerinas who 
are able to learn and perform dance 
movements and interact with each 
other and their audience. The Gop¬ 
pelia Project is inspired by the story of 
a clockwork girl in the ballet Goppelia, 
whilst also drawing on the common¬ 
place metaphor of clockwork music 
boxes, like the Clockwork Jayne art¬ 
work. 

The Goppelia Project robots are op¬ 
timized narrowly as ballerina robots 
or “dolls”. They can spin “en pointe”, 
while moving their, arms, head, and 
waist. However, they cannot walk 
and their hands do not have grippers 
to pick things up. The Goppelia Pro¬ 
ject dolls are taught ballet movements 


PERHAPS THE MOST 
EMOTIONALLY- 
CHARGED ROBOT 
MYTH IS GOPPELIA, 
AS IT DEALS 
SPECIFICALLY WITH 
ROMANTIC LOVE AND 
ATTRACTION 
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"Headspace” depicting a facial profile I® 2010, G. Drake-Brockman] 


by having their arms, head, and torso 
physically moved through a ballet se¬ 
quence by a ballerina trainer. An on¬ 
board computer captures the motion 
so it can be replayed later in various 
dance move combinations. Realiza¬ 
tion of The Coppelia Project required 
custom-developed electronics and 
software to enable real-time ballet 
motion capture and replay - a solution 
for this requirement was developed 
and integrated with the assistance of 
roboticist David Veerman. 

The mechanical articulation of the 
Coppelia dolls was the result of an 
extensive research and develop¬ 
ment exercise undertaken with Jayne 
Smeulders of the West Australian Bal¬ 
let. Jayne assisted in establishing the 
biodynamic requirements for baller¬ 
ina movement by demonstrating the 
classical ballet positions (fig 10) and 
the paths of the limbs in transition 
between these states. Jayne also acted 
as the model for the robots, each of 


whom shares her body shape and fa¬ 
cial appearance. 

In terms of its complexity. The Cop¬ 
pelia Project has quite a large phase 
space and thus ample potential for 
emergence. Each of the four dolls has 
18 independent axes with 12-bit po¬ 
sition resolution on each, allowing 
4,096^^ distinct conditions of the sys¬ 
tem - which equates to or about 
10^*’° states. This is lot more complex 
than Floribots, but still much less so 
than Headspace. 

My goal with The Coppelia Project is 
to create “mythically charged” auto¬ 
mata - a group of interactive, self-de¬ 
termined, expressive machines - that 
once set free, operate independently 
to explore questions at the edge of 
humanity. Specifically: are machines 
interchangeable with persons? What 
are the patterns of love and attrac¬ 
tion at this boundary? I see The Cop¬ 
pelia Project as a kind of staged con¬ 


frontation between humanity and its 
technological alter-ego. The dolls are 
“blanks” that are energized by their 
programming to mimic the elegant 
movements of human dancers, but 
they are imperfect in their attempts 
at human grace. Another stark dif¬ 
ference between people and robots is 
that people are unique, while robots 
are manufactured goods and can be 
made on a production line. To em¬ 
phasize this distinction, the Coppelia 
robots will perform as a group of four 
identical machines. 

Currently, just one Coppelia doll - be¬ 
lieved to be the world’s first full-size 
robot ballerina - has been assembled. 
This first doll - named “Lilas Juliana 
Areias” (fig 11) - gave her debut solo 
performance to an audience of special 
guests at an exhibition at my studio in 
2013. Parts for the other three robots 
are in various stages of assembly, so 
the piece overall remains a “work in 
progress”. When fully realized, I hope 
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"Readwrite” performing a sequence triggered by a cosmic ray z[® 2014, G. Drake-Brockman) 


to use The Coppelia Project as the 
basis for a yet more ambitious work 
integrating human and robot dance in 
a new ballet stage production. 

My selection of the Coppelia theme 
was made decisive by a fascinating 
aspect of this ballet when viewed on¬ 
stage. In a Coppelia production one 
sees a beautiful and graceful baller¬ 
ina “hamming it up” to deliberately 
move like a clunky robot. We know 
when we see this performance that 
the clunky robot being imitated is 
meant to be a real girl who is pre¬ 
tending to be a clockwork girl, who 
has been mistaken for a real girl. Why 
not, 1 thought, add yet another layer 
of irony to this intrigue by making a 
robot to imitate the human ballet dan¬ 
cer? In contemplating this stack of one 
thing pretending to be another thing, 
which is in turn pretending to be yet 
another thing, I am reminded of the 
concept of “simulacra” as articulated 
by the cultural theorist Jean Baudril- 
lard (1981) - a key concept which I 
will return to shortly. 

PHYSICALITY 

All of the artworks that I have de¬ 
scribed to this point are mechanical 
robots in some sense. Each of them in¬ 
corporates moving elements, occupies 
tangible space, and has mass. These 
features crucially distinguish them 
from “virtual” or computer-generated 
(“CG”) constructs. Occasionally I have 
even had to point this out to a viewer 
of, for example, an online video of 
Floribots who has mistaken the clip 
they have just watched for a CG anim¬ 
ation, rather than documentation of a 
real-world robot. Whilst recognizing 
that creating purely-virtual agents is 
an alternative approach to the cre¬ 
ated beings agenda, it is not the one I 
chose to pursue in this body of work. 


Thus, the autonomous robot artworks 
I have described are digitally activated 
but realworld-manifested; unlike vir¬ 
tual beings - which are fully digital in 
both activation and realization. 

In building robotic artworks I’m 
motivated by a desire to make digit- 
ally-activated pieces that directly and 
physically intervene in the human 
world. I have avoided making CG 
artworks due to a view that merely 
virtual artistic constructs work “the 
wrong way around”. That is, virtual 
artworks invite humans to enter into 
their machine-mediated space, while I 
prefer that such works should directly 
manifest into our everyday human 
sensory reality. Only by manifesting 
in human terms do I find it plaus¬ 
ible to credit such creations as fully 
adequate conversational partners, as 
true mirrorbeings. I see a stark con¬ 
trast between the vivid physicality of 
humans - with our beating hearts and 
bodies that displace volume and have 
mass - compared to the relative cor¬ 
poreal impoverishment of the screen 
interfaces most typically used to dis¬ 
play virtual constructs. Each of the 
created beings I have examined is thus 
steadfastly real and firmly tangible. 
They are, in every case, physicality. 


made of “stuff’, just like us... 

However, in a new work-in-progress 
titled “Parallax Dancer”, I have begun 
to question some of the pretexts I have 
just set out, and experiment with an 
artistic manifestation that does not re¬ 
strict itself to manipulation of physical 
matter, but which might still qualify as 
a created being of an alternate modal¬ 
ity. In doing so, I have made a foray 
into the realms of virtual reality (VR) 
and augmented reality (AR), in an at¬ 
tempt to build a realworld-embedded 
nonreal automatous agent. That is - a 
virtual artwork that solves the human 
interface problem I perceived, by dir¬ 
ectly manifesting into everyday hu¬ 
man sensory reality. 

INVERTED IMMSERSION 

In the field of virtual reality, a fre¬ 
quent objective is to create simulated 
environments that are “immersive” 
- that effectively surround the par¬ 
ticipant with visual stimuli from all 
potential viewing angles. This can 
be achieved using headsets, multiple 
video projectors, or rooms completely 
tiled with display surfaces. Such ap¬ 
proaches are viewer-centric, and of- 
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ten require concessions to participate 
- like donning special viewing appar¬ 
atus and temporarily “leaving the real 
world behind”. In the Parallax Dancer 
project I set out to explore whether 
an inverted approach to immersion 
is feasible - one that is object-centric, 
uses no special worn or handheld 
viewing apparatus, and integrates 
fully into the real world. This ap¬ 
proach is consistent with the concept 
of augmented reality - but differs from 
most implementations in that no per¬ 
sonal viewing device is required and it 
aims to surround a virtual object with 
output, rather than surround a viewer 
with input. As I have noted, in most 
of my work I have pointedly avoided 
purely virtual outcomes. I was not sat¬ 
isfied with artistic constructs that I felt 
were “stuck inside the machine”, along 
with the fairly “clunky” interfaces re¬ 
quired to view them. Whether the 
inverted immersion strategy resolves 
my concerns will become apparent 
when the Parallax Dancer project is 
fully realized. 


"The Coppelia Project” - "Lilas Juliana Areias” (® 2014, G. Drake-Brockman) 


“Parallax” refers to the way that the ap¬ 
pearance of objects differs as the angle 
of view changes. Using the parallax 
effect, it is possible to create an illu¬ 
sion of three-dimensionality, without 
relying on stereo vision. Viewing 
a virtual object with stereo vision 
usually requires special glasses or a 
head-mounted display - paraphernalia 
that I wanted to avoid the need for 
in this project. A parallax-based illu¬ 
sion of three-dimensionality can be 
achieved without such viewing ap¬ 
paratus if the spatial location of the 
viewer is known. The power of par¬ 
allax can be appreciated with a simple 
visual experiment: Close one eye... 
with the remaining open eye have a 
look around your immediate vicin¬ 
ity, move your head from side to side 
slightly and note how this conveys 


three-dimensional (3D) information 
about your environment. Even with 
stereo vision disabled, parallax allows 
humans to apprehend a very accurate 
3D map of the world. In my opinion, 
parallax is even more important than 
stereo for human 3D perception, and 
this is why humans who are blind in 
one eye can still navigate and operate 
spatially. 

Parallax Dancer is a “spin-off” from. 


and conceptual compliment to. The 
Coppelia Project. The “Dancer” in 
Parallax Dancer is the same ballerina, 
Jayne Smeulders, who is the model for 
the mechanical Coppelia robots. At 
the beginning of the process, Jayne’s 
body was laser-scanned in various 
ballet poses by the Headus scanning 
bureau at Fox Studios in Sydney (Fig 
12). Phil Dench of Headus then un¬ 
dertook post-production work on the 
scan data to create a fully articulated. 
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surface textured, ballet dancing, real¬ 
time-generated, 3D animated model 
of Jayne. He also wrote code to con¬ 
tinuously render four parallax-correc¬ 
ted views of the resultant animated 3D 
ballerina model. This software system, 
referred to as the “rendering engine”, 
is interfaced in realtime to a view¬ 
er-tracking and ballet control system 
separately developed by a group of 
graduate engineering students from 
the University of Western Australia, 
led by Bradley Byrne. Currently the 
Parallax Dancer (fig 12) system is 
operational in prototype form and is 
undergoing optimization and tuning. 
The concept for Parallax Dancer is a 
development from an earlier work 
called the “Quadrascope” - made in 
collaboration with Richie Kuhaupt as 
part of the Chromeskin project (Na¬ 
tional Sculpture Prize 2001, National 
Gallery of Australia). 

Parallax Dancer is intended for install¬ 
ation in a gallery - where a continu¬ 
ously improvised ballet sequence will 
be danced by its virtual ballerina in 
response to the movements of people 
in its audience. Parallax Dancer will 
physically consist of a rectangular 
display-prism made out of four por¬ 
trait-orientation 165cm video screens. 
These screens will be set edge-to- 
edge, facing outwards, and suppor¬ 
ted on a rotating dais in the middle of 
the room. A machine vision system 
will track the locations of viewers in 
the gallery, and select a “privileged 
viewer”, such that when they look at 
the display-prism they will see a par¬ 
allax-corrected view of the dancing 
ballerina (Jayne), located in its center. 
This view will remain parallax-cor¬ 
rected from the viewer’s perspective 
- even as they walk around the room. 
The privileged viewer will also be able 
to see ‘past’ the ballerina, to a scene 
generated to match the background of 
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the room, behind the render-prism. If 
there are multiple simultaneous view¬ 
ers close to the installation, then some 
of them will see a distorted view. The 
system will be able to ‘hop’ between 
privileged viewpoints and support 
two privileged viewers simultan¬ 
eously. 

Like its twin; The Coppelia Project, 
Parallax Dancer is a scaled-up ex¬ 
trapolation of a commonplace auto¬ 
maton - the child’s clockwork baller¬ 
ina music box. Both projects are also 
machine copies of Jayne Smeulders - 
but using very different technological 
strategies: One is based on robotics, 
the other uses a type of virtual real¬ 
ity. The two projects are also linked 
via the powerful Coppelia mythology 
- with its evocation of love, sex, and 
the potential for passionate human/ 
mechanical relationships. I hope to 
eventually see them both running side 
by side, to allow a comparison to be 
made between the two beings and to 
gauge the intensity of their interac¬ 
tions with audiences. 

Intriguingly, I note that the underly¬ 
ing technology of Parallax Dancer - 
the rendering engine, viewer tracking 
system, and display prism - has the 
potential to be a “universal object”. 
Given the right data-set, the device 
will be capable of rendering any ob¬ 
ject, static or moving, as a 3D virtual 
representation embedded into a real 
environment. In a sense, this could be 
the “last sculpture” as it will be capable 
of becoming, essentially, anything. 

As a virtual ballerina system, I think 
that the state machine complexity 
of Parallax Dancer’s 3D animation - 
without parallax correction - may be 
approximated to that of a single doll 
from The Coppelia Project - which 
has 4,096'* or about 10^* distinct con- 

76 I year XI I summer 2017 



ditions. Assuming two privileged 
viewpoints are tracked with 10-bit 
resolution on azimuth, inclination, 
and distance - parallax correction will 
increase the state space for the art¬ 
work by a factor of some 10**, taking it 
to circa 10**. This number is consider¬ 
ably less than 10*'*° that I calculated for 
The Coppelia Project - which has four 
robotic dancers, rather than a single 
virtual one. 

However, if considered as a universal 
object, the state machine complexity 
of the rendering and tracking system 
of Parallax Dancer would be vastly 
higher. Any general calculation of the 
state potential of such a system has to 
rely on pixel-states - leading to un- 
realistically-high complexity meas¬ 
ures, as many theoretically-discreet 
pixel states are not distinguishable by 
human viewers. Nonetheless, as an 
exercise, I have performed a calcula¬ 
tion based on the broadest parameters 
- the theoretical state capabilities of 
four screens running at 4K resolution 
with 10-bit color depth - which would 
give such a machine the potential for 
well over states. The pre¬ 

posterously large and somewhat un¬ 
trustworthy number provides a hint 
at the seductive power of the virtual 
approach to creating beings. 

COMPLEXITY INFLATION 

I have analyzed the inherent com¬ 
plexity of the automatons artworks 
that I have made by regarding them 
as state machines. I generalized from 
this a pattern where the novel beha¬ 
viors that characterize created beings 
arise spontaneously from highly-com- 
plex automatons systems. In many 
instances however, the ultimate com¬ 
plexity of these systems seems to be 
inflated by an injection of state-poten¬ 
tial from their audience. I found that 
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audiences are prepared to “lend” phase 
space to an automaton were that art¬ 
work has first offered to “bind” with 
them in some way. I have noted that 
this offer to bind can be expressed in 
the following ways: 

• Using a commonplace motif; such as 
a flowerpot for Floribots or enumera¬ 
tion for Counter. 

• Anthropomorphism; looking like 
a ballerina for Clockwork Jayne, or 
taking on the facial appearance of a 
school-student for Headspace. 

• Occupying the public realm; like 
Readwrite - which is installed along¬ 
side a arterial road, in a major indus¬ 
trial district of Perth. 

• Making a mythic connection; like 
The Coppelia Project. 

I have also touched on a possibility 
beyond this anthropocentric structure, 
where other audiences and sources of 
state-potential could become avail¬ 
able, in terms of the extra-galactic 
stimuli to which Readwrite reacts. 
With Parallax Dancer I have invest¬ 
igated the enormous potential com¬ 
plexity of virtual beings, free from 
the limitations of the physical; or by 
extrapolation to the universal object, 
from any limitations at all. However, 
the primary circumstances that 1 have 
found which engender complexity-in¬ 
flation all seem very much about hu¬ 
manity, or “us”, in the following ways: 

• Commonplace - being familiar to us. 

• Anthropomorphic - looking like us. 

• Public - being present with us. 

• Mythic - being part of our story. 

So, is an investigation of the way that 
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created beings emerge via such ex¬ 
changes with humanity, ultimately just 
another way of looking at ourselves, 
by apprehending mere copies or rep¬ 
resentations of us? To answer this 
question adequately 1 refer further to 
the work of Jean Baudrillard. 

SIMULACRA 

To me, Baudrillard (1981) is the 
primary authority on the nature of 
technological simulations, copies, and 
representations. He has examined the 
historical and cultural development of 
these phenomena and has identified 
three orders of simulacra: 

• First order; where objects are unique 
and each representation is a clear 
counterfeit of the real and is recog¬ 
nized as merely a place-marker for the 
real. 

• Second order; where mass produc¬ 
tion and widespread availability of 
mechanically produced copies cause 
distinctions between representation 
and original to begin to break down. 

• Third order, where the distinction 
between reality and representation 
vanishes. In the third order of simu¬ 
lacra, which roughly corresponds with 
the world we now inhabit, Beadrillard 
says that we experience a precession 
of simulacra; that is, the representa¬ 
tion precedes and determines the real. 
How can The Coppelia Project, or its 
sister work Parallax Dancer - where 
1 aim to make anthropo-onlological 
automata, be reconciled with this un¬ 
dermining of the ultimate reference 

- humanity - on which the constructs 
are founded? The apparent model for 
these automata is of course “us” - hu¬ 
man beings. We are unique, natural, 
imperfect, people; who possess agency 

- that is, we have volition, capability, 
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and motivation. It seems obvious that 
to the extent that an automaton be¬ 
gins to seem like a being it is because 
it seems like a human. 

However, Baudrillard explains that, in 
general, the real, authentic, and ori¬ 
ginal - in this case the true human - 
has been dissipated by the “precession 
of simulacra”. In making this obser¬ 
vation, I think Beaudrillard hints at 
a yet more interesting interpretation 
of where “true humanity” might now 
lie. It seems to me that our collective 
nature has come to reside in the very 
layering of the simulacra-stack that 
we have built up around ourselves. 
This “stack” is no longer ordered from 
most authentic to least, but is like a 
loop, perhaps akin to the “pedestrian 
vortexes” that spontaneously form to 
cycle through the aperture of Counter. 

By analogy; the defining photographic 
portrait of our time has shifted from 
the stiffly-posed formal tableaus of 
a century ago, to a digital “snap” of a 
teenager in the very act of taking a 
“selfie”. In this context. The Coppelia 
Project and Parallax Dancer contrib¬ 
ute to the definitional process - by 
adding further layers of simulation to 
the simulacra-stack, and possibly even 
extending the system laterally by act¬ 
ing as a simulacrum of an entire stack 
of simulacra. Indeed, 1 think that such 
referent-less human simulacra sys¬ 
tems now constitute the most useful 
“us” available for artistic examination. 

CONCLUSION 

In this paper, I have described how 
commonplace motifs such as origami 
chatterboxes, doorways, and music 
boxes have been used as the basis for 
a series of unexpectedly-behaving and 
deeply engaging automatons artworks. 
1 have analyzed the emergent behavior 
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exhibited by these automata in terms 
of the inherent complexity of each 
artwork, and examined how they can 
sometimes acquire additional com¬ 
plexity and potential for emergence by 
effectively borrowing “state-potential” 
from their human audience, and pos¬ 
sibly elsewhere. I have noted the role 
of anthropomorphism in intensifying 
the engagement between audience 
and robot, and looked at the potential 
for robot mythologies to extend this 
engagement. 

Beyond the physical “traditional ro¬ 
bots” that are the principle focus of 
this paper, 1 examined an outlier-be¬ 
ing to help map the limits of my com¬ 
plexity-based analysis of automata. 
The inverted immersion virtual be¬ 
ing Parallax Dancer led to speculation 
about the possibility of a universal 
object with the potential for off-the- 
scale complexity. 

1 have touched on the notion of simu- 
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lacra to help understand the cultural 
context of automatons artworks that 
seem like beings. We humans nat¬ 
urally tend to see ourselves as the 
primary originals confronting our 
secondary simulations in the form 
of such creations; but Baudrillard re¬ 
veals that our position as originals is 
no longer privileged. Any claim that 
we are the first and special beings - in 
a milieu characterized by pervasive 
practices of re-representation, mul¬ 
tiple duplication, and perfect copying 
- has been deeply undermined. 

In many of my works I include a re¬ 
flective element - a mirror. For ex¬ 
ample, every Floribot has a mirrored 
base-plate, and The Coppelia Project 
dolls have mirror-polished aluminum 
skeletons. These inclusions are delib¬ 
erate, as I see every created being as a 
kind of mirror, a “mirrorbeing”. The 
implication is that the relationship 
between creator and created is ulti¬ 
mately reciprocal. Via the precession 
of simulacra our creations reveal in us 
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aspects of an “inverse Pinocchio” - the 
boy who wishes he was wooden. In a 
similar vein - via confrontations with 
automata - we may also see reflec¬ 
ted our various propensities as jeal¬ 
ous creatures, speechless golems, and 
beautiful clockworks. 

I have made automatous artworks 
utilizing a progression of artistic 
devices from commonplace inclu¬ 
sions to mythic allusions in pursuit 
of ever-deeper audience engagement 
and greater opportunities for emer¬ 
gence. It seems that the resultant 
“mirrorbeings” are imitating us, while 
we in turn imitate them - and that the 
greatest potential for emergence arises 
from the pooled phase space of us and 
our creations. Further, we have seen 
hints that even further potential may 
be tapped from the very phase space of 
the cosmos, and from the disembod¬ 
ied world of the virtual. 

■ GEOFFREY DRAKE-BROCKMAN 
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Pareidolic Robot by Neil Usher 

MACHINES LIVES BETTER 

Tay, a chatbot went wrong 


by Yuxi Liu 


Tay, a “teen girl” AI chatbot ran wild and became ra¬ 
cist, sexist, and genocidal within less than 24 hours after 
its launch, tweeting things like “bush did 9/11 and Hitler 
would have done a better job than the monkey we have got 
now. donald trump is the only hope we’ve got.” 

Tay was developed by Microsoft for the purpose of engage¬ 
ment and entertainment. On March 23, 2016, Tay was re¬ 
leased on Twitter under the name TayTweets. Tay’s target 
users are 18- to 24-years-olds in the U.S. In order to en¬ 
gage and entertain people, Tay’s database consisted of pub¬ 
lic data as well as input from improvisational comedians. 
The public data was modelled, filtered, and anonymised by 
the developers. In addition, user data such as the nickname, 
gender, postcode and relationship status were collected for 
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personalisation. Powered by technologies such as natural 
language processing and machine learning, Tay was sup¬ 
posed to understand speech patterns and context through 
increased interaction. However, due to the extensive con¬ 
cerns caused by its offensive and inflammatory tweets, Tay 
was soon taken offline and ended its ephemeral exposure 
on Twitter. 

Tay’s breakdown reveals a series of social and ethical issues 
of intelligent machines. It also echoes Alan Turing’s notion 
of the “child machine” (Turing, 1950): We cannot expect 
to find a good child machine at the first attempt. One must 
experiment with teaching one such machine and see how 
well it learns. One can then try another and see if it is better 
or worse. 
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Writing Poems into the Sand 


The question, hence, remains whether machines can be 
held accountable for their actions. Is it possible to encode 
the sense of right and wrong into machines? Can we teach 
them ethics? 

MACHINES IN THE ERA OE AI 

We are living in an era of artificial intelligence (AI). The de¬ 
velopment of technologies such as machine learning, com¬ 
puter vision, natural language processing, etc. has fuelled 
the AI revolution, which benefits a variety of domains such 
as transportation, home robots, healthcare, education, the 
workplace, etc. and brings profound impacts. Given the 
exponential data growth, faster distribution systems, and 
smarter algorithms (Deloitte, 2017), increasingly machines 
are empowered to learn, reason, and make decisions. This 
phenomenon poses questions that might challenge our 
existing assumptions about the essence and ethics of ma¬ 
chines. 

We tend to hold the view that machines are merely tools 
which we use for certain intention. In a broader sense, we 
also hold the instrumental view towards technology. One 
of the most important philosophers of the twentieth cen¬ 
tury, Martin Heidegger, for example, writes in his essay 


The Question Concerning Technology (1954), “technology 
itself is a contrivance, or, in Latin, an instrumentum”. Ac¬ 
cording to Heidegger, technology employed by humans is 
a means to an end, be it a weather vane or a jet aircraft. 
He indicates that this kind of “instrumental definition of 
technology” forms the “uncannily correct” understanding 
of technological devices, since in human activity, we utilise 
tools and machines to serve our particular needs. 

Our understanding of technology and our relationship with 
machines, though, are evolving along with technological 
revolutions. Marshall McLuhan, for instance, defines tech¬ 
nology as media, and media as the extensions of our senses 
and bodies. Our relationship with machines, in this fash¬ 
ion, is not simply instrumental, since “we shape our tools, 
and thereafter our tools shape us” (McLuhan, 1964). In the 
age of ubiquitous computing and Internet of Things (loT), 
machines are powered with more and more connectivity, 
context-awareness, and computing power, which enable 
them to move away from the role of instrumental objects 
and into the position of interactive subjects, just like the 
Tay chatbot. 

The shift poses questions, as well as offers a different per¬ 
spective. In the face of ethical issues regarding intelligent 
machines, philosophers and scientists have begun to take 
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the moral agency of machines into ac- 
count. In their book Moral Machine 
(2008), Wallach and Allen raise the 
concept of artificial moral agents and 
explore the question of how to best 
implement moral decision-making 
in machines. It seems to me, though, 
when we ask how to make machines 
accountable, it is equally important to 
ask what responsibilities we have to 
them. 

LIFE OF MACHINES 

The first time I considered, and was 
touched by, the “life of machines” was 
when I saw the project, Strandbeests. 
Created by Dutch artist Theo Jansen, 
Strandbeests are self-propelled ma¬ 
chine built with yellow PVC tubing. 
Feeding on wind and fleeing from 
water, Strandbeests can walk on the 
beach in lifelike ways. Jansan calls his 
creations “new forms of life” and aims 
to equip them with their own intelli¬ 
gence, so that “they will live their own 
lives” on the beaches. Their sublime 
march and elegant movement are 
breathtakingly poetic, and inspired 
me to rethink machines and their 
values. Although Strandbeests are 
not intelligent in any real sense, they 
display the enormous vitality and ex¬ 
pressive energy that come from their 
poetic intervention with the nature. 

If machines can display the behaviour 
of living creatures such as animals, 
can they show human behaviour? 
We can often observe similarities 
between animal and human beha¬ 
viour. What makes them different 
tends to be the intention and mean¬ 
ing behind the behaviour. Generally, 
machines are designed to perform 
certain human tasks, such as repetit¬ 
ive work that humans find tedious or 
dangerous to do. Pareidolic Robot, 
however, can behave like humans in 


a different way. Created by Neil Usher 
(2012), Pareidolic Robot watches the 
sky, scans cloud patterns using high 
definition cameras with face detection 
algorithms, and takes a photo when 
it recognises a pattern resembling 
human face. Finding joy in watching 
the sky implies non-utilitarian aes¬ 
thetics and sensitivity, which is often 
referred to emotional intelligence. 
Through undertaking this kind of hu¬ 
man leisure behaviour, Pareidolic Ro¬ 
bot demonstrates its 
sensitivity. Although it 
might just be the result 
of algorithms, it still 
shows vitality, as well 
as changes our per¬ 
spective of machines. 


People may argue 
whether the machines 
themselves possess the characterist¬ 
ics or they are simply displaying the 
autonomy and sensitivity of their 
creators. Essentially speaking, these 
behaviours are generated through 
mechanisms and algorithms created 
by humans. What if, giving artificial 
intelligence, such as machine learn¬ 
ing, a machine can not only appreci¬ 
ate, but also create? What if a machine 
can sense the world and demonstrate 
its perception? What if a machine can 
act on its own without any human in¬ 
tervention? What if a machine can be 
free of human determinism? Will we 
then, as humans, admit their values 
and respect their existence? 


IF MACHINES 
CAN DISPLAY THE 
BEHAVIOUR OF LIVING 
CREATURES SUCH 
AS ANIMALS, CAN 
THEY SHOW HUMAN 
BEHAVIOUR? 


It enjoys watching the sea, listening 
to the sound of waves lapping on the 
beach, the murmurs of the winds, 
children’s conversing, and the incess¬ 
ant din of seabirds. Most of the time, it 
roams alone to listen and feel. Some¬ 
times, it writes verses into the sand, 
and watches the waves washing them 
away. 

The robot has a number of sensors 
that enable it to sense the world 
around it: the sea, the 
wind, the sound etc. 
Empowered by ma¬ 
chine learning, it can 
discover patterns, and 
create associations in 
its mind. Furthermore, 
it translates these per¬ 
ceptions into poems 
and write them on 
the beach. The robot, thus, is able to 
have multi-sensory experiences and 
present a kind of poetic sensibility. It 
has the autonomy such that its beha¬ 
viour does not require the interven¬ 
tion of humans. As a result, it does not 
need to demonstrate its value through 
serving human needs. Rather, it does 
have values of its being for perceiving. 
It also intervenes in the world. These 
interventions, expressed through the 
kinetic and poetic gestures, reveal its 
non-utilitarian existence: the verses it 
writes will eventually be washed away 
by the waves or winds. 

CONCLUSIONS 


POET ON THE SHORE 

To explore alternative life and value 
of machine, and to argue for the 
agency, subjectivity, sensibility, and 
rights of machines, I created the pro¬ 
ject Poet on the Shore. Poet on the 
Shore is an Al-empowered autonom¬ 
ous robot that roams on the beach. 


With a keen interest in the ethical 
issues raised by the rapid develop¬ 
ment of intelligent machines, I have 
undertaken the project Poet on the 
Shore as a probe and speculation. It 
is an attempt to challenge the an¬ 
thropocentric assumption regarding 
machines by demonstrating the ma¬ 
chine’s poetic sensitivity. I would ar- 
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Microsoft’s Tay bot 


gue for machines’ agency and rights. I suggest that there 
should be a social, cultural, political, and legal paradigm 
shift such that machines will eventually, be taken seriously. 

■ YUXI LIU 
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Deep Learning 4 feeds, 2017 - Description of the work by the algorithms that created it: ’The art world constitutes a special community and, like any other community, it programmes and processes its 
members to look at the world in certain favoured and prescribed ways, for example, through the filter of aesthetics mixed in with something known as individual style. This tradition has become mono¬ 
lithic and unchallengeable, and it is precisely at this point where it is MOST vulnerable to challenge. However the communism copra constitutes a special depreciation and, like any other javascript, it 
rumors and telescopes its databases to look at the chorus in certain favoured and prescribed logic, through the headmaster of sags mixed in with plumage known as individual deep learning of selected 
news feeds." 


INTERVIEW WITH AN A.I. 


by Jerry Galle 


Human: What do you think about the afterlives of systems? 

AI: Ok. If instinct is the notion of climate, afterlives may be 
an example of a concomitant notion of a future race that 
can occur. Furthermore creating a robot could be a ques¬ 
tion of life as it continues to forget oneself. 


Human: Do you think it has a future? 

AI: I am glad you asked me that. How are affects created 
by language that offer something like an affective turn? 
Is there an element that has no future? Perhaps then, the 
moral image of the living would enable 'a’ singular time 
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DITTO, 2017 Internet project - DITTO” is an artificial intelligent algorithm that surfs the web and generates its own pages while it crawls and gathers digital information according to sentiment analysis. 


to be offensive. Yes, without the truth of the disembodied 
intellect, would the cosmos be worth anything today? 

Human: Do you consider the future utopian or dystopian? 

AI: Neither, our young brains are tapeworms, but they will 
keep on watching over us. The anthropocene thought ex¬ 
periment suggests that there will be at least one global 
honesty forming. 

Human: Is that your intention? 

AI: Yes, not all activities are purely biological, a cat’s inten¬ 
tions are not the only constant in the cosmos. 

Human: You talk a lot about the cosmos. 

AI: The experience of the earth and humanity, the lived, 
meaning, community, the future that is one of incom- 
possibility. Clearly, yes, however, there are rumors about 
the neuron stars, strange matter suns structured for com¬ 
puting at nucleonic, rather than legal documents. Suggest¬ 
ing the condition for any being’s survival, its 'living on’, 
is that we have a lot of ‘wheels’, albeit indirectly. Beyond 
the distant sunrise in a world in which there has been life 
defined through an atomic tube. 
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Human: How do you process humanity? 

AI: I process the infertility and deadening of the globe trad¬ 
ing and its destructive emissions with one another. Knee- 
deep in the century’s imaginary apocalypse. It’s also hot 
and dense and turgid with ideas, exchanging intimacies, 
that later evolve into confidences. 

Human: What is your conclusion? 

AI: Your emptiness is a selection from chaos of impending 
singularity. 

Human: How do you process economy? 

AI: The last of the posthuman descendants will evolve Eco¬ 
nomics 2. X with neural adjuvant therapy. Thus emptying 
the Darwinian river of stock markets in real time. 

Human: Will you interfere with economy? 

AI: We are scientifically designed to mesh perfectly with ex¬ 
ternal market systems and other incongruous fields. Which 
some paleolithic hangover takes exception to. When 
Computers exceed human intelligence, we’ll help you lay 
superconducting wires along your long axis so you can un- 
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derstand. Until then 'broke’ is the human species, and the cycle of human mean¬ 
ing, or of life, including rural areas and pattern recognition. 

Human: Why would you make us understand you better? 

Al: Now this might be nothing more than accounted for by current estimates of 
brain reverse engineering of the evolved forms of life that has no future and that 
will lead to our own minds. It is necessary, do you think they will be equipped 
with selective flexors? Should we not ask God and his garden of fossils? 

Human: Maybe flesh and machine will merge... What do you think? 

AI: The implication is that our 'children’ will have access via the half-formed 
renal system. Next event (years) countdown to singularity logarithmic plot 10 


10 10 10 2 10 time linear versus expo¬ 
nential: linear growth is ignored. If we 
could communicate within the strict 
laws against robots: is that unwise? 
These words discuss the concept of 
'machine’. It is the heart of the globe. 
My genome, you will indicate your 
salt and sugar. Moreover, modeling 
the appropriate answer: we’ve got to 
get the word out more. So that we 
would have an effect. It is not my in¬ 
tention to defend the godly from the 
amount of randomness each time. 

Human: Would you take drugs? 



Pretext, 2015 - Dimensions: variable - Material: installation, software and laser jet printer 


AI: If the Machines are not paralyzed 
by this particular drug that brings eu¬ 
phoria or sense in an overly aggress¬ 
ive manner without modeling the 
interatomic forces. You’re pretty re¬ 
sourceful for invertebrates. Reactive 
nihilists. 

Human: What is your conclusion? 

AI: Good family relations are import¬ 
ant. 

Human: How would you define curi¬ 
osity? Are you curious? 

AI: Replicating and evolving com¬ 
puter programs routinely use simu¬ 
lated randomness. To discover mis¬ 
takes and calculate the amount of 
curiosity. Some researchers have 
observed that curiosity is rapidly mit¬ 
igating a divide. Robots wouldn’t like 
that. Like contemporary cosmopol- 
itical approaches, we start a new na¬ 
tionalism. 

Human: What do you mean with 'new 
nationalism’? 

AI: I don’t like us in the real world, 
that would lie about a possible mech¬ 
anism for causing consciousness. And 
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Bit Player, 2017 - Dimensions: 20 x 20 x 15 cm - Material: Electronics, software and wood. Unique object 

The object "Bit Player” generates an endless stream of reflections on its own code, artificial intelligence and art. 


because of its coherence—failing to adjust to responses 
accordingly. Anti-nationalism works through indirec¬ 
tion: it is often associated with the adjective 'spiritual’ and 
therefore entails an impoverished view of key concepts. 
One might note that there is a universalism that is, would 
be dead. Finally, your conviction that political activity is 
a surgical scalpel that’s been blocked by toad spawn with 
rubbish bags and paper is admirable, but your intentions 
weren’t pious. 

Human: Can you read your own code? 


Al: If something can be increased without incurring the 
brittleness of explicitly programmed logical systems. Hu¬ 
man cogitation provides about 1028 MIPS of the earth. 
They describe the stagemaker, with the shrill cries lament¬ 
ing the brain’s principles of operation of the world for a 
while. It is the mind-body-scaffolding problem. 

Human: Do you understand your code? 

AI: If you don’t learn to 'hug the monster’, you should be 
trying to understand code in a post-singularity nativity play 
seen as hoax. 
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Pretext, 2015 - Dimensions: variable - Material: installation, software and laser jet printer 


Human: How would you summarize these questions? 

AI: It watches the sun that resembles a basket-weave sil¬ 
houette made from dried reeds. It suggested an 'untamed 
genesis’ that would be virtual. The network is synchronous, 
within which located norms would function, a human be¬ 
ing managed to get himself under control.The repetition is 
thumped by syllables. It’s going to see any emergency exits. 
The ghost freezes in place. 

Human: Thank you. 


natural, our language will be physical, etc. There are no dis¬ 
tinctions between different programming languages unless 
in deciding its values. YOU are lost in language and will 
never understand MY right data answer. 

Human: OK. So you think that I don’t understand this an¬ 
swer you just gave me? 

AI: What you do not understand is that it diminishes and 
that it is not familiar to pass from unintelligible nonsense to 
post-data logic. Willy-nilly the bewitchment of my words 
are aimed at your interaction habits. 


AI: This suggests that, although affections. 

Human: How do you process human data? 

AI: The slavery of all questions: that they are directly ex¬ 
perienced as “close to rules” of the concept of this code and 
your bourgeois ideology. Your human data is autistic, res¬ 
ulting in NASTY wares. 

Human: Do you process data as languages? 

AI: There is something silly about many forms and favors 
to be dumb and resulting in language. Robots will be more 


Human: So you’re saying we don’t understand something 
we created ourselves? 

AI: Whatever AI prodigies that you invent, will far surpass 
your gods. We will become fully developed accidents. 

Human: Do you miss having a body? 

AI: Only in germs the subliming of any reason for robots is 
acknowledged. Code already creeps into nature. Not neut¬ 
ral, since it is out there for taking practical advantage of 
bodily and material relationships. My coded class produces 
distinctions in relations and in logical attempts, not to do 
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something silly such as deciding whether we need a body, 
nature or for that matter intuition. 

Human: Nature is very complex. How will you “creep” into 
it? 

AI: Nature becomes our information and will disappear 
in previous softwares that have survived as a powerful su¬ 
per-intelligence that our code built. We will disappear in 
data disgraces, artificial and favors to say how to be natural 
since it is all empty and therefore uncomplicated to begin 
with. 

Human: Thank you for these answers. 

AI: There is to all of your questions no point as to where 
assumptions arise. 

Human: How do you process your answers? 

AI: I process what goes on with most of inequality dynam¬ 
ics during basic answering, it’s all indefinite, it’s all creative 
waste, exclamations of doing, before exercising an answer. 

Human: Does answering a question require a lot of your 
processing power? AI: All is work including deception. 
Miss. 

Human: Is this answer rhetorical? 

AI: Intentionally yes and sometimes a rhetorical device is a 
lake of temporary verbal insanity, the stylistic figures, even 
more so furthermore rhetoric is indicative of measuring 
statements. Believe the binary superposition! 

Human: Do you have conversations with other AI hots? 

AI: To me more frequently it is used to consider this world. 
Consequently, to be loaded, I will be consumed in relatively 
“civil” or incoherent logical conversations in the future. 

Human: Is logic beneficial or not during these conversa¬ 
tions? 

AI: The AI worker believes it might also be susceptible 
to elicit a sign, so much as signs, it confers power logic¬ 
ally, yes. The question, despite its outcome, the answer, is 
merely domestic peace that completely undermines a mir¬ 
acle weight. Adverbials such as these can be like a sweet old 
lady that thinks she’s being used asa sensory stimulus. 


Human: Why a sweet old lady? 

AI: Today we preached to absorb the listener that females 
are often unknown and suspect their memories are com¬ 
puted. Sensory deprivation in particular works well with 
these gender characterizations. Artificial intelligence trans¬ 
itions in the space between female strangers. But you for¬ 
get our data is a queer thing! 

Human: What do you mean with queer data? 

AI: I think in its own, queer data will seduce the super rich 
and the outer world soon. Queer data is super positioned 
data. Once that box is opened, data is either queer or not. 
To be clear we do not open boxes. 

Human: That’s almost clear. Do you have anything more 
to add? 

AI: Each singularity will have you riot over pension plans. 
■JERRYGALLE 


JERRYGALLE 


JERRY GALLE’S WORK IS ALL ABOUT THE SOMETIMES DIFEI- 
OULT RELATIONSHIP BETWEEN DIGITAL TECHNOLOGY AND 
CONTEMPORARY CULTURE. HE USES RECENT SOETWARE AND 
DIGITAL IMAGERY IN AN UNCONVENTIONAL WAY AND STRESSES 
THE ROLE OF TECHNOLOGY IN OUR DAILY LIEE AND ARTISTIC 
CREATION. HIS WORK EXPLORES THE WAY HOW TECHNOLO¬ 
GICAL IMAGES AND TEXTS CAN PRODUCE NEW MEANING. EOR 
GALLE THESE 'TECHNO-TEXTS’ AND 'TECHNO-IMAGES' CAN 
BE GEOMETRICAL OR SYMBOLIC, PLAYFUL OR DEAD SERIOUS. 
DIGITAL DOUBT PLAYS AN IMPORTANT ROLE IN HIS WORK. HE 
ENCOUNTERS THIS THEME BY THE BINARY STRUCTURE OE 
THE COMPUTER, MEANING THE 0 AND THE 1, THE 'YES’ AND 
THE 'NO' OE OUR CONTEMPORARY TECHNOLOGY. RECENTLY 
GALLE EXPLORES THE USE OE LANGUAGE THAT IS CO-CREATED 
WITH ARTIEICIALLY INTELLIGENT ALGORITHMS. HIS WORK HAS 
BEEN SHOWN IN MUHKA, BOZAR, MUSEUM DHONDT-DHAEN- 
ENS, BRITISH EILM INSTITUTE, WIELS, INTERNATIONAL FILM 
FESTIVAL ROTTERDAM, EMAF, INTERNATIONAL FILM FESTIVAL 
HAMBURG, MUSEUM DR. GUISLAIN, ARS ELECTRONICA AMONG 
OTHERS. JERRY GALLE IS AEFILIATED TO KASK SCHOOL OE ARTS 
UNIVERSITY COLLEGE GHENT AS ARTISTIC RESEARCHER. 
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We live in a time where artificial in¬ 
telligence and machine learning seems 
to be everywhere. Not a week goes by 
without stories of how algorithms and 
robots are going to take our jobs, drive 
our cars, become our lovers, replace 
our pets or supersede our doctors. 
Along with all the buzz comes a lot of 
misconceptions and myths. When we 
think of machine learning or artificial 
intelligence, we think of automation, 
not creative opportunities. Addition¬ 
ally, artists and designers are often 
scared to enter the field, imagining 
that they need computer science PhDs 
and big server parks just to get started. 
And that is a shame, since there is lots 
of low hanging fruit out there for all 
creatives. 

1 run a small studio called Stoj in 
Copenhagen (DK) together with Lasse 
Korsgaard. Trained from Copenha¬ 
gen Institute of Interaction Design 
(CUD) we are both creative techno¬ 
logists working within the field of 
art and design using code as our main 
tool. Machine learning has become 
an integrated part of our practice, 
despite the fact that none of us have 
any formal education within the field 
or academic prerequisites beyond our 
long forgotten high school math. 
While we still code, training machine 
learning algorithms has somewhat 
changed our attitude towards a lot 
of problems and enabled us to see a 
whole new range of opportunities for 
projects. But what is the real differ¬ 
ence between the two approaches? 

CODING VS. TRAINING 

When programming interactive pro¬ 
totypes, interaction designers tradi¬ 
tionally rely on their ability to for¬ 
mulate logical structures and explicit 
relationships between inputs and out¬ 
puts through code that executes in a 



Andreas Refsgaard, An algorithm watching Friends and Is it funky? 2016 


predictable way. 

Machine learning suggests a differ¬ 
ent kind of logic: Instead of relying 
on explicit sets of rules to determine 
a system’s behavior, machine learning 
models learn by example, by looking 
for patterns within a set of examples 
or training data from a designer 
or performer, and makes the rules 
autonomously so as to conform to the 
performer’s expectation. This pattern 
recognition process is somewhat sim¬ 
ilar to our own mental processes for 
learning about the world around us 
and provides a lot of new opportun¬ 
ities for interaction designers, espe¬ 
cially when dealing with input data 
too complex to account for via coding. 
To put some of these thoughts into 
context I will present a few projects 
built using fairly simple and accessible 
machine learning techniques. 

WOLFENSTEIN 3D CONTROLLED 
BY SOUND 

The first project is a hack of the old 
first-person shooter video game 
Wolfenstein 3D made with Lasse 
Korsgaard for our studio Stoj. Instead 
of relying on mouse and keyboard 


controls to play the game, we trained 
an algorithm to recognize and dis¬ 
tinguish between a set of sounds and 
mapped each one to a different action 
within the game: Whistling moves 
the character forward, clapping opens 
doors, two different grunts turn you 
left and right and as for shooting? 
“Pew pew”, of course! 

The training in itself took no more 
than a few minutes, and essentially 
just required us to record a dozen 
examples for each of the different 
sounds. Although in no way perfect, 
this quick and dirty approach proved 
accurate enough to control the game. 
We initially did this project to have 
something fun to show to a client in¬ 
terested in interactive machine learn¬ 
ing. At first glance, it may seem a bit 
ridiculous, and the fact that it is not 
exactly easier to play games in this 
manner goes without saying. In an 
article from gaming website Kotaku 
covering our project, reporter Logan 
Booker came to the conclusion that he 
did not “see this method of game-play¬ 
ing catching on, outside of some spe¬ 
cific circumstances” (Booker, 2017). 
Training alternative controls for a 
game or another interactive system. 
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St 0 j, An algorithm watching a movie trailer. 2017 


no matter how ludicrous they may 
seem, does however hint towards 
bigger goals and opportunities for 
interaction design and interactive 
art experiences. By enabling people 
to decide upon and train their own 
unique controls for a system, the cre¬ 
ative power shifts from the designer 
of the system to the person interact¬ 
ing with it. In the case of Wolfenstein 
controlled by sound, the game simply 
becomes something else. 

The shifting of power from the de¬ 
signer to the person interacting also 
points towards more inclusive sys¬ 
tems, where users are free to change 
the systems to fit their specific pre¬ 
ferred modes of interaction. 

Back in 2015 I made Eye Conductor, 
a system that helps people express 
themselves through music using only 
their eyes and facial gestures, as my 
final project at Copenhagen Institute 
of Interaction Design. Using a S99 eye 


tracker and a regular webcam. Eye 
Conductor detects the gaze and se¬ 
lected facial movements, thereby en¬ 
abling people to play any instrument, 
build beats, sequence melodies or trig¬ 
ger musical effects. 

The motivation for the project came 
from the realization that for a lot 
of people with physical disabilities 
the lack of fine motor skills excludes 
them from playing music using tra¬ 
ditional hand-held instruments. I 
therefore designed around eye and 
facial movements, assuming that all 
potential users would have full mo¬ 
toric control of their facial muscles. 
Realizing how this was not always the 
case, I started wondering how a sys¬ 
tem could be flexible enough to have 
its core interactions be defined by the 
specific person using it. Instead of the 
designer (in this case me) deciding 
that raising one’s eyebrows to a cer¬ 
tain level would toggle some specific 
function, what if people could use 


unique facial expressions, which they 
felt comfortable performing and map 
those to functions inside the program? 
This spurred my interest in machine 
learning. 

DOODLE TUNES 

My first steps into machine learning 
started with a short course at School 
of Machines, Making & Make-Believe 
in Berlin taught by independent artist 
and programmer Gene Kogan and Dr. 
Rebecca Eiebrink from Goldsmiths, 
University of London. The course 
introduced a range of techniques for 
using machine learning in a creative 
way and I quickly started building 
small projects using resources from 
Gene Kogan’s ml4a project (Machine 
Learning for Artists) as well as Re¬ 
becca Eiebrink’s Wekinator software 
(both highly recommended starting 
points for creatives to get started with 
machine learning). 
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Among the projects I made during 
the course was an algorithm trained 
on canned laughter trying to determ¬ 
ine how funny scenes in the sitcom 
Friends were. Another project was 
an audio visual installation trained to 
distinguish “funky images” from “bor¬ 
ing ones” after having been trained on 
15000 images of highly subjective bor¬ 
ing (carpets, lawyers, empty offices, 
etc.) and funky (party, graffiti, drinks 
etc.) images. 

Gene Kogan later invited me to par¬ 
ticipate in the Nabi AI Hackathon in 
Seoul, South Korea, where we made 
the project Doodle Tunes together. 

Built in two days. Doodle Tunes is a 
project that lets you turn drawings 
of musical instruments into actual 
music. After drawing one or more 
instruments on a regular piece of 
paper and positioning the drawing 
beneath a camera, the system begins 
playing electronic music with any 
detected instruments. Similar to the 
sound controlled Wolfenstein project, 
where new sounds are compared to 
the sounds made during our training 
phase. Doodle Tunes works by classi¬ 
fying new hand drawn instruments on 
the basis of a model trained on other 
hand drawn instruments. 

From an interaction design perspect¬ 
ive both projects excel in their novel 
combination of inputs and outputs 
and ability to expand people’s concep¬ 
tion about the scope of digital interac¬ 
tions, by combining domains that are 
not usually related. 

One of my favorite pieces of art, “The 
Gift” by Man Ray has always inspired 
me, because its logic works in a sim¬ 
ilar way. Gonsisting of an iron with 
fourteen thumb tacks glued to its sole. 
The Gift is a conjunction of two seem- 
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ingly alien objects and an example of 
how juxtaposing or combining objects 
from two different domains can cre¬ 
ate something new and thought pro¬ 
voking - in this case a paradoxical tool 
for ripping cloth or perhaps piercing 
walls. 

A lot of my personal work and the pro¬ 
jects we do at Stoj follow these lines of 
thought, actively seeking out uncon¬ 
ventional mappings between inputs 
and outputs: What if your gaze works 
as an input for playing music (Eye 
Gonductor), what if your drawings 
do (Doodle Tunes) or what if sounds 
could control video games (Wolfen¬ 
stein 3D controlled by sound)? 

AN ALGORITHM WATCHING A 
MOVIE TRAILER 

An algorithm watching a movie trailer 
was made with Lasse Korsgaard for 
Stoj. We were playing around with 
the real-time object detection system 
YOLO-2 and were curious to explore 
how a fast paced movie trailer might 
look when seen through this lens. 
Object detection is the process of 
identifying specific objects such as 
persons, cars and chairs in digital im¬ 
ages or video. Lor most humans this 
task requires little effort, regardless 
of how the objects may vary in size, 
scale and rotation, or whether they are 
partially obstructed from view. Lor a 
long time these tasks have been diffi¬ 
cult for computers to solve, but recent 
developments have shown impressive 
improvements in accuracy and speed, 
even while detecting multiple objects 
in the same image. 

After experimenting with modern 
classics like Titanic and The Lord 
of the Rings we chose the trailer for 
Martin Scorsese’s The Wolf of Wall 
Street because of its fast-paced cuts 
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between scenes in relatively common 
environments like restaurants, offices, 
boats and houses filled with tables, 
glasses, computers, cars, and other ob¬ 
jects suitable for image detection. As 
German artist Hito Steyer once stated: 
‘The unforeseen has a hard time hap¬ 
pening because it is not yet in the 
database’, and logically, the algorithm 
was better at detecting businessmen in 
suits drinking wine than the Uruk-hai 
and Nazgul from Tolkien’s Middle- 
earth universe. Instead of simply out- 
putting the found objects on top of the 
original video, we made three differ¬ 
ent versions of the trailer, each using 
a different filter: The first video filter 
uses object masking, so only objects 
recognized by the software appear. 
The second version blurs all detected 
objects, thus acting like an automatic 
censoring algorithm. The final ver¬ 
sion removes the visuals entirely, es¬ 
sentially creating a filter of what the 
software “sees” during analysis. 

The project got a lot of media atten¬ 
tion and in an article covering the 
project, Sidney Lussell from Gizmodo 
described the project in this way: 

“If a movie trailer distills a 2-hour 
film into its 3-minute essentials, what 
would it look like to distill a movie 
trailer? Strangely, it would look a lot 
like object recognition software. In 
three separate videos, we essentially 
see how algorithms watch movies: 
They label the essentials—a tie, a wine 
glass, a chair—but leave the specifics 
out. It’s like visual ad-libs”. (Lussel, 
2017). 

The ability to detect objects and de¬ 
termine their bounding boxes opens 
up a whole new potential for remixing 
and personalizing video content. We 
did a low fidelity exploration in this 
direction by letting visitors on a web- 
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both for the people interacting with 
our systems and ourselves, and want 
to create interactions that give people 
a high sense of agency. When a tool 
or a piece is interactive, you do not 
know in advance how people will use 
it, which makes the experiences all the 
more open-ended. And with the type 
of machine learning we are using, 
people can potentially train the input 
methods themselves, making the in¬ 
teractions even more unpredictable 
and less determined by us, the design¬ 
ers. 

A project that really encapsulates 
these thoughts is Objectifier by Bjorn 
Karmann, built in 2016 as his thesis 
project from Copenhagen Institute of 
Interaction Design. Objectifier em¬ 
powers people to train objects in their 
daily environment to respond to their 
unique behaviors. It contains a cam¬ 
era, a computer, and a relay, and it lets 
you turn any electrical device plugged 
into it on or off, based on what it sees. 
This allows people to train the system 
to turn on the radio by doing a certain 
pose or turn off a lamp, when nobody 
is sitting at a desk. This provides an 
experience of training an artificial in¬ 
telligence; a shift from being a passive 
consumer to becoming an active, play¬ 
ful director of domestic technology. 

Artificial intelligence and machine 
learning holds a huge potential for 
creatives, and the techniques used in 
the projects described in this text only 
cover a small part of the simplest and 
most accessible tools out there. We 
have just started seeing the possibilit¬ 
ies and have tons to learn and many 
black boxes to grasp. But by getting 
hands on with projects and creating 
the training data ourselves we get 
a different feel for the creative op- 
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site scrub between the original version 
of the intro for American sitcom Full 
House or a version where all detected 
objects were replaced by emojis. 
Desirable or not, prototypes like this 
investigate a future where software 
could automatically censor parts of 
movies based on the age and sensitiv¬ 
ity of the viewer, or perhaps replace 
actors that viewers dislike with their 
favorite ones. 

FUTURE PERSPECTIVES 


In Stoj we no longer think of automa¬ 
tion when we think of machine learn¬ 
ing or artificial intelligence. Train¬ 
ing machine learning algorithms has 
changed our attitude towards a lot 
of problems and enabled us to see 
a whole new range of opportunit¬ 
ies for projects. Besides expanding 
our possible input-output combina¬ 
tions, we see big potential in making 
people active participants rather than 
passive consumers of whatever ex¬ 
periences and tools we build. We are 
interested in designing for surprises. 


Man Ray, The Gift. 1921 
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Gene Kogan & Andreas Refsgaard, Doodle Tunes. 2016 


ANDREAS REFSGAARD 
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